Download - Using real-time dashboards to improve student engagement in virtual learning environments
Using real-time dashboards to improve student engagement in
virtual learning environmentsRobert Bodily, Steven WoodBrigham Young University
Research questionsDashboard DevelopmentR1: What technical requirements are needed for an online learning system to collect and provide students with personalized information in a student dashboard?R2: What functionality do students want in a dashboard and how should it be visually represented?Data Mining AnalysisR3: What is predictive of student success in a first-year general chemistry course?R4: Can we develop an early alert warning system using clickstream data?
R1: Technical Requirements
Challenges with current LMS
• A lot of learning is not occurring within a Learning Management System (LMS)
• Interoperability standards
• No access to real-time data• Canvas data (1 day old)• API (rate limiting factors)
• Do not track enough data• No information on how students interact with a page
Our analytics system
LTI (Learning Tools Interoperability)Single sign-on system for learning applications and learning management systems
xAPI (Experience API/Tincan API) Data format specification for data management interoperability
LRS (Learning Record StoreDatabase that stores xAPI statements and provides real-time data access
R2: Functionality
Student Dashboard
Challenges:
Low student use
Initially not useful for students
Students had too many things to do already
Low engagement with feedback
Instructor dashboard
R3: Predictive elements of student success
Context
• Class• First year chemistry course• Blended – class 3x per week,
• Resources• 150 videos (avg. 2 min long, supplemental resources)• 15 weekly quizzes (unlimited question attempts)
• Participants• 200 students (online interactions)• 96 students took the self-report resource use survey
Data collected• Quiz
• Confidence in answer (just a guess, pretty sure, very sure)• Time spent on quiz• Correct/incorrect• Number of attempts per question• Leave tab (still open, but inactive), come back to tab (active again)
• Video• Play, pause, skip forward/backward, change play rate, change volume,
• Dashboard• Number of times students follow recommendations given in dashboard• Number of clicks within the dashboard
What course elements are predictive of student success?
Variable Beta P-valueOnline homework score 0.366 0.000In-class IClicker scores 0.154 0.024# of attempts/question -0.411 0.000Amount of question navigation -0.206 0.040# of online activity sessions -0.195 0.020
Variable Beta P-valueRead the textbook 2.443 0.059Ask professor questions in class 7.363 0.000Watch Khan Academy -2.738 0.051Use the internet -3.199 0.010Skip recitation -4.820 0.041
Model 1 – regressing online interaction data on final exam score.
Model 2 – regressing self-report resource use on final exam score.
R4: Early alert warning system
Develop an early course prediction of student achievement
Online student interaction data Online student interaction data AND exam scores
There is significant improvement in both models until week 3 or 4, so that seems to be a good time to make predictions for instructors and students.
Thank you! Questions?