Download - Final atp2017 smarter proctoring
#atpconf#atpconf
Frequencies of Academic Integrity Violations Across Multiple Proctoring Modalities
Dr. Mac Adkins – CEO & Founder, SmarterServicesRick Beaudry – CEO, B Virtual
WK Conference : testing2017
#atpconf
ABOUT
the
PRES
ENTE
RAbout the Presenters …
■ Dr. Mac Adkins, CEO & Founder, SmarterServices§ Test Publisher- SmarterMeasure Learning Readiness Indicator – Taken by 4 million students.
§ SmarterAssessments – Adaptive Assessment Engine§ SmarterProctoring – Multi-modal Proctoring Management
■ Rick Beaudry, CEO, B Virtual§ UPCEA Corporate Industry Advisory Council
3/6/17 2
#atpconf
Research in Progress
3/6/17 3
■ Kennesaw State University§ Dr. Michael Goodwin, Academic Integrity Coordinator§ Since 2011 over 11,000 students have taken SmarterMeasure§ 372 matches of students with academic integrity violations§ Statistical analysis to identify non-cognitive factors that are
indicators of likelihood to engage in academic integrity violations- Time management, - Availability of Time, - Motivation, - Procrastination, - Technology Skills, - Keyboarding skills, etc.
#atpconf
Abhorrent Behavior Frequencies - 90 day
3/6/17 5
Unsecure environment24%
Missing ID16%
Unapproved Items13%
Invalid ID10%
Unable to Read ID8%
Communicating with Others
7%
Eye Movement5%
Bio Break4%
Blackboard/Whiteboard4%
Unable to view face during exam
4%
Reading out loud3%
Standing up2%
#atpconf
Peaks and Valleys of Virtual Proctoring
3/6/17 6
Virtual Proctoring Monthly Variations in Volume
#atpconf
Top 4 Reasons for Technical Failures
3/6/17 7
■ 1. Connectivity Issues ■ 2. Audio issues ■ 3. Webcam issues ■ 4. Exam site/LMS issues ■ 5. Aberrant behavior issues
#atpconf
2016 Technical Failure Rates
3/6/17 8
Row Labels Sum of CountAdobe Acrobat Failure 3Audio issues 205Bomgar 1279Customer exam site 694Exam Connection 245Internet/bandwidth issues 836Java Failure 7Megameeting 801Password Issue 146Scheduling software 5Skytap issues 14Technical Issues 50Technology -‐ Audio 78Technology -‐ Bandwidth 174Technology -‐ Browser (IE Settings) 25Technology -‐ Disconnect Bomgar 827Technology -‐ Disconnect MM 433Technology -‐ Webcam 245Web Browser 104Webcam 498Grand Total 6669
#atpconf
% refunds for wait time issues in 2016
3/6/17 9
■ January - 1.40%■ February - 0.84%■March - 1.62%■ April - 3.94%■May - 1.53%■ June - 1.53%■ July - 1.60%■ August - 1.40%■ September - 1.31%■October - 1.00%■ November - 1.30%■ December - 1.15%
#atpconf#atpconf
Test Your Proctoring Prowess
Results of the 2016 Annual Proctoring Survey
WK Conference : testing2017
#atpconf
3/6/1711
What comes to mind when I say …
Testing Integrity
On your phone/laptop go to:
Menti.com 21 04 90
#atpconf
About The Survey
3/6/17 12
■ 35 day period in the Spring of 2016. ■ During this time 365 responses were received.
This is an increase from 284 the prior year.■ Each stakeholder group was presented with
questions specific to their role.
#atpconf
Common Proctoring Modality
■Which proctoring modality was reported by faculty as most commonly used?§ A – Approved Human Proctor§ B – Instructor as Proctor§ C – Local Test Center§ D – Virtual Proctoring
#atpconf
Common Proctoring Modality
■Which proctoring modality was reported as most commonly used?§ A – Approved Human Proctor§ B – Instructor as Proctor§ C – Local Test Center§ D – Virtual Proctoring
#atpconf
Common Learner Authentication
■ The most common form of learner authentication reported was:§ A – Key stroke monitoring§ B – User name / password§ C – Photo ID Scanning§ D – Security Questions
#atpconf
Common Learner Authentication
■ The most common form of learner authentication reported was:§ A – Key stroke monitoring§ B – User name / password§ C – Photo ID Scanning§ D – Security Questions
#atpconf
Satisfaction with Proctoring Modalities
■With which modality of proctoring did facultyreport the lowest level of satisfaction?§ A – Approved human proctor§ B – Virtual proctoring§ C – Automated Virtual Proctoring§ D – Corporate Testing Center
#atpconf
Satisfaction with Proctoring Modalities
■With which modality of proctoring did faculty report the lowest level of satisfaction?§ A – Approved human proctor§ B – Live, Virtual proctoring§ C – Automated Virtual Proctoring§ D – Corporate Testing Center
Note that they were most satisfied with an approved human proctor.
#atpconf
Psychological Deterrent
■ According to faculty, which proctoring modality was the highest level of psychological deterrent to cheating.§ A – Instructor proctored§ B – Approved human proctor§ C – Virtual proctoring§ D – Local test center
#atpconf
Psychological Deterrent
■ According to faculty, which proctoring modality was the highest level of psychological deterrent to cheating.§ A – Instructor proctored§ B – Approved human proctor§ C – Virtual proctoring§ D – Local test center
#atpconf
Psychological Deterrent
■ According to students, which proctoring modality was the highest level of psychological deterrent to cheating.§ A – Instructor proctored§ B – Approved human proctor§ C – Test Center§ D – Live, virtual proctoring
#atpconf
Psychological Deterrent
■ According to students, which proctoring modality was the highest level of psychological deterrent to cheating.§ A – Instructor proctored§ B – Approved human proctor§ C – Test Center§ D – Live, virtual proctoring
#atpconf
Threat to the Quality of eLearning
■What percentage of faculty considered issues related to proctoring a high level of threat to the reputation of the quality of eLearning. § A – 24%§ B – 32%§ C – 35%§ D – 39%
#atpconf
Threat to the Quality of eLearning
■What percentage of faculty considered issues related to proctoring a high level of threat to the reputation of the quality of eLearning. § A – 24%§ B – 32%§ C – 35%§ D – 39%
Learner authentication was also considered a high threat.
#atpconf
Cost of Proctoring
■What is the average cost for a one-hour proctored testing session?§ A - $15§ B - $20§ C - $25§ D - $40
#atpconf
Cost of Proctoring
■What is the average cost for a one-hour proctored testing session?§ A - $15§ B - $20§ C - $25§ D - $40
#atpconf
Proctor’s Improvements
■What would proctors like to see improved in the proctoring process?§ A – Assignment scheduling§ B – Communications with learner§ C – Proctoring protocols§ D – Proctoring technology
#atpconf
Proctor’s Improvements
■What would proctors like to see improved in the proctoring process?§ A – Assignment scheduling§ B – Communications with learner§ C – Proctoring protocols§ D – Proctoring technology
#atpconf
Proctor Training
■ How did most proctors rate the training they have received for serving as a proctor?§ A – Very weak§ B – Weak§ C – Strong§ D – Very Strong
#atpconf
Proctor Training
■ How did most proctors rate the training they have received for serving as a proctor?§ A – Very weak§ B – Weak§ C – Strong§ D – Very Strong
#atpconf
Important Issues
■What are the most important issues to School Administrators relative to testing integrity?§ A – Cost to the learner§ B – Cost to the institution§ C – Learner authentication§ D – Proctoring policies
#atpconf
Important Issues
■What are the most important issues to School Administrators relative to testing integrity?§ A – Cost to the learner§ B – Cost to the institution§ C – Learner authentication§ D – Proctoring policies
School leaders are least concerned with proctoring technologies.
#atpconf
Payment of Proctoring Fees
■When a fee is charged for proctoring services who is most likely to pay?§ A – School pays§ B – Learner pays§ C – Learner and school share the expense§ D – Depends on the situation
#atpconf
Payment of Proctoring Fees
■When a fee is charged for proctoring services who is most likely to pay?§ A – School pays§ B – Learner pays§ C – Learner and school
share the expense§ D – Depends on the situation
#atpconf
Preferred Proctoring Modality
■Which proctoring modality is most preferred by students?§ A – Approved human proctor§ B – Local test center§ C – Virtual proctoring§ D – Instructor proctored
#atpconf
Preferred Proctoring Modality
■Which proctoring modality is most preferred by students?§ A – Approved human proctor§ B – Local test center§ C – Virtual proctoring§ D – Instructor proctored
#atpconf
Difficulty to Cheat
■ According to students, in which proctoring modality would it be most difficult to cheat?§ A – Approved human proctor§ B – Corporate testing center§ C – Location such as a library§ D – Virtual proctoring
#atpconf
Difficulty to Cheat
■ According to students, in which proctoring modality would it be most difficult to cheat?§ A – Approved human proctor§ B – Corporate testing center§ C – Location such as a library§ D – Virtual proctoring
#atpconf
Test Center Volume
■ In the prior year the volume of tests proctored in collegiate testing centers:§ Substantially decreased from prior
years§ Decreased from prior years§ Remained about the same as prior
years§ Increased over prior years§ Substantially increased over prior
years
#atpconf
Test Center Volume
■ In 2015 the volume of tests proctored in collegiate testing centers:§ Substantially decreased from prior
years§ Decreased from prior years§ Remained about the same as prior
years§ Increased over prior years§ Substantially increased over prior
years
#atpconf
Spectrum of Proctoring Modalities
Lecture Hall / Instructor as
ProctorCampus
Testing CentersApproved Proctoring
Professionals
Monitoring and/or
Restricting Technologies
Live, Virtual Proctoring
Biometric Learner
Authentication
Automated or Non-‐live Virtual
Proctoring
SmarterProctoring is a Holistic Solution that Manages All Of These Modalities.
#atpconf
Software that works within your LMS that helps students find different types (Virtual, Testing Center, Human, etc.) of approved proctors, while also allowing the faculty/administrators to see where each student is in the test scheduling process and securely communicating all test details (passwords, allowable resources) to the proctor.
What is SmarterProctoring?
Proctoring Process Management System
#atpconf
Comments, Questions
■ Dr. Mac AdkinsCEO & [email protected]
■ Rick BeaudryCEOB [email protected]