yorke hallam wardman evaluation of an electronic marking ... · schoolofdentistry faculty of...

School of Dentistry FACULTY OF MEDICINE AND HEALTH EVALUATION OF AN ELECTRONIC MARKING SYSTEM FOR AN UNDERGRADUATE DENTAL OSCE ASSESSMENT Yorke V.C., Hallam J. and Wardman M.J. OSCE is a high stakes clinical assessment and the systems used to mark, record and process candidate performance data must, as well as being robust, be efficient to use by both examiners and administrative staff. OSCEs at the School of Dentistry, University of Leeds • Typically, 100 students take OSCEs at each sitting, with 14-16 stations • Optically read paper sheets have traditionally been used for marking • Significant administrative time has been needed to collate, check and process approximately 1400 papers • Marks have not been available to analyse immediately meaning that release of marks and delivery of feedback is not as timely as required by educational quality standards. A new electronic system of marking for OSCE has recently been introduced in Leeds with the aim of improving efficiency. Examiners and administrative staff may have concerns that electronic systems are not as reliable as traditional paper formats and about the transition to a technology based system Introduction Conclusion Aims of the Study Materials and Methods Results . To evaluate the introduction and use of an electronic marking system for an undergraduate dental OSCE • To determine the adaptability of, and ease of use for, examiners • To gain an insight into the experiences of examiners and administrators in the use of the new marking system Training in the use of an electronic marking system (using video and verbal presentation) was provided to all staff who examined a year 4 undergraduate dental programme OSCE when the new system was introduced in 2017. A questionnaire was distributed to all examiners (39) and administrative staff (6) following completion of the OSCE. University of Leeds Ethics Research approval was gained 130117/JH/221. https://www.youtube.com/watch?v=l5Rjy3dY0A4 Marking system training video • Both examiners and administrative staff adapted easily to the implementation of a new electronic marking system and would choose to use the system again. • Previous and current use of technology should be considered when deciding on training requirements. • This study may be of interest to other dental schools considering the implementation of technology based OSCE marking systems. ”I thought the system was great! The training video was very good, and simple to follow.”- Examiner “Despite my initial hesitation, it was a very positive experience- I think we should continue to use this system as it makes all our lives easier!!”- Administrator 80% of examiners and 75% of administrative staff found the electronic marking system easy to use. Current and previous use of technology was found to be influential in the need for future training: Overall, 90% of examiners and 85% of administrative staff had a positive experience using the system and 100% of all staff surveyed would choose to use the system again.

Upload: vanduong

Post on 24-May-2018

213 views

Category:

Documents

0 download

Report

Download

Embed Size (px):

TRANSCRIPT

School of DentistryFACULTYOFMEDICINEANDHEALTH

EVALUATION OF AN ELECTRONIC MARKING SYSTEM FOR AN UNDERGRADUATE DENTAL OSCE ASSESSMENT

Yorke V.C., Hallam J. and Wardman M.J.

OSCEisahighstakesclinicalassessmentandthesystemsusedtomark,recordandprocesscandidateperformancedatamust,aswellasbeingrobust,beefficienttousebybothexaminersandadministrativestaff.

OSCEsattheSchoolofDentistry,UniversityofLeeds

• Typically,100studentstakeOSCEsateachsitting,with14-16stations

• Opticallyreadpapersheetshavetraditionallybeenusedformarking• Significantadministrativetimehasbeenneededtocollate,check

andprocessapproximately1400papers• Markshavenotbeenavailabletoanalyse immediatelymeaningthat

releaseofmarksanddeliveryoffeedbackisnotastimelyasrequiredbyeducationalqualitystandards.

AnewelectronicsystemofmarkingforOSCEhasrecentlybeenintroducedinLeedswiththeaimofimprovingefficiency.Examinersandadministrativestaffmayhaveconcernsthatelectronicsystemsarenotasreliableastraditionalpaperformatsandaboutthetransitiontoatechnologybasedsystem

Introduction

Conclusion

Aims of the Study

Materials and Methods

Results

ToevaluatetheintroductionanduseofanelectronicmarkingsystemforanundergraduatedentalOSCE

• Todeterminetheadaptabilityof,andeaseofusefor,examiners• Togainaninsightintotheexperiencesofexaminersand

administratorsintheuseofthenewmarkingsystem

Trainingintheuseofanelectronicmarkingsystem(usingvideoandverbalpresentation)wasprovidedtoallstaffwhoexaminedayear4undergraduatedentalprogramme OSCEwhenthenewsystemwasintroducedin2017.Aquestionnairewasdistributedtoallexaminers(39)andadministrativestaff(6)followingcompletionoftheOSCE.UniversityofLeedsEthicsResearchapprovalwasgained130117/JH/221.

https://www.youtube.com/watch?v=l5Rjy3dY0A4Marking system training video

• Bothexaminersandadministrativestaffadaptedeasilytotheimplementationofanewelectronicmarkingsystemandwouldchoosetousethesystemagain.

• Previousandcurrentuseoftechnologyshouldbeconsideredwhendecidingontrainingrequirements.

• ThisstudymaybeofinteresttootherdentalschoolsconsideringtheimplementationoftechnologybasedOSCEmarkingsystems.

”I thought the system was great! The

training video was very good, and simple to follow.”-

Examiner

“Despite my initial hesitation, it was a

very positive experience- I think we should continue to use

this system as it makes all our lives

easier!!”- Administrator

80%ofexaminersand75%ofadministrativestafffoundtheelectronicmarkingsystemeasytouse.Currentandprevioususeoftechnologywasfoundtobeinfluentialintheneedforfuturetraining:

Overall,90%ofexaminersand85%ofadministrativestaffhadapositiveexperienceusingthesystemand100%ofallstaffsurveyedwouldchoosetousethesystemagain.

7013540r4 8000DH Direct Heat Incubators · Marking of electrical and electronic equipment, which applies to electrical and electronic equipment falling under the Directive 2002/96/EC

Electronic Plan Solutions - Lincoln, NebraskaElectronic Plan Solutions MARK - Marking up in ProjectDox ® 9.x MARK - Marking up in ProjectDox 9.x Rev. 2018-09-04 2 CONTENTS 1 About

3M Mini D Ribbon (MDR) Connectors · Certain Hazardous Substances in Electronic Information Products. This symbol, per Marking for the Control of Pollution Caused by Electronic Information

3M Dynatel Electronic Marking System (EMS) Marker ... Dynatel Electronic Marking System (EMS) Marker Locator 1420-iD Operator’s Manual 1420-iD Marker Locator 1420E-iD Marker Locator

Electronics Coding and marking of electronic components · the electronics market. The challenge: Electronic components frequently require codes that fit in small, constrained spaces

GradeMark for Electronic Marking and Feedback Denis Parkinson & Dr Stephen McKinnell School of Health Sciences

SLOPE Final Conference - electronic marking of trees

Marking Guide...2013/06/06 · Marking guide. June 2013 [electronic resource] ISBN: 978-0-7711-5424-9 1. Mathematics —Examinations, questions, etc. 2. Educational tests and measurements—Manitoba

Dentistry Ethics Teaching Theme Team M.J. Wardman, R.G. Baker, C. Potter LDI

3M Mini-Clamp Socket - Heilind ElectronicsCertain Hazardous Substances in Electronic Information Products. This symbol, per Marking for the Control of Pollution Caused by Electronic

Guide to e/E-Marking - Elite Electronic Engineering, Inc. · Guide to e/E-Marking for EMC Type Approval of Vehicles and Electronic Sub-Assemblies Provided by Elite Electronic Engineering,