lessons learned from moodle vle/lms data in the field
TRANSCRIPT
![Page 1: Lessons Learned from Moodle VLE/LMS Data in the Field](https://reader036.vdocuments.mx/reader036/viewer/2022062306/5a666bdb7f8b9a5d3d8b4b63/html5/thumbnails/1.jpg)
Lessons from Moodle Data
“Dr. John” WhitmerDirector, Analytics and Research
MoodleMoot UK/I | 12-April 2017
![Page 2: Lessons Learned from Moodle VLE/LMS Data in the Field](https://reader036.vdocuments.mx/reader036/viewer/2022062306/5a666bdb7f8b9a5d3d8b4b63/html5/thumbnails/2.jpg)
1. Learning Analytics Overview & Bb Data Science
2. Research & Modeling Findings
1. Virtual South Carolina Moodle Predictive Model
2. Differences in Student Achievement by Tool Use
3. Discussion
![Page 3: Lessons Learned from Moodle VLE/LMS Data in the Field](https://reader036.vdocuments.mx/reader036/viewer/2022062306/5a666bdb7f8b9a5d3d8b4b63/html5/thumbnails/3.jpg)
Learning Analytics Overview
![Page 4: Lessons Learned from Moodle VLE/LMS Data in the Field](https://reader036.vdocuments.mx/reader036/viewer/2022062306/5a666bdb7f8b9a5d3d8b4b63/html5/thumbnails/4.jpg)
Educational Technology Assessment Hierarchy
Does it impact student learning?
(Learning Analytics)
How many people use it? (Adoption)
Does it work? (SLAs)
![Page 5: Lessons Learned from Moodle VLE/LMS Data in the Field](https://reader036.vdocuments.mx/reader036/viewer/2022062306/5a666bdb7f8b9a5d3d8b4b63/html5/thumbnails/5.jpg)
What is Learning Analytics?
Learning and Knowledge Analytics Conference, 2011
“ ...measurement, collection, analysis and
reporting of data about learners and their
contexts, for purposes of understanding
and optimizing learning
and the environments
in which it occurs.”
![Page 6: Lessons Learned from Moodle VLE/LMS Data in the Field](https://reader036.vdocuments.mx/reader036/viewer/2022062306/5a666bdb7f8b9a5d3d8b4b63/html5/thumbnails/6.jpg)
Techniques
• Simulation if X, what Y? (“With this Ultra Learning Analytics trigger rule, how many students would trip notified?”)
• Hypothesis testing: investigate if a specific relationship is true (“What’s the relationship between time spent in a course and student grade”?)
• Data mining: analyze underlying latent patterns in data (“What typical patterns in tool use characterizes BB Learn courses?”)
Key Data Sources
• Moodlerooms & X-Ray
• Learn Managed Hosting & SaaS
• Collaborate Ultra
Main Big Data Sources & Techniques
![Page 7: Lessons Learned from Moodle VLE/LMS Data in the Field](https://reader036.vdocuments.mx/reader036/viewer/2022062306/5a666bdb7f8b9a5d3d8b4b63/html5/thumbnails/7.jpg)
Commitment to Privacy & Openness
• Analyze data records that are not only removed of PII, but de-personalized (individual & institutional levels)
• Share results and open discussion procedures for analysis to inform broader educational community
• Respect territorial jurisdictions and safe harbor provisions
![Page 8: Lessons Learned from Moodle VLE/LMS Data in the Field](https://reader036.vdocuments.mx/reader036/viewer/2022062306/5a666bdb7f8b9a5d3d8b4b63/html5/thumbnails/8.jpg)
Virtual South Carolina Online
![Page 9: Lessons Learned from Moodle VLE/LMS Data in the Field](https://reader036.vdocuments.mx/reader036/viewer/2022062306/5a666bdb7f8b9a5d3d8b4b63/html5/thumbnails/9.jpg)
Feature Importance in Predictive Model
![Page 10: Lessons Learned from Moodle VLE/LMS Data in the Field](https://reader036.vdocuments.mx/reader036/viewer/2022062306/5a666bdb7f8b9a5d3d8b4b63/html5/thumbnails/10.jpg)
Predictive Accuracy for Risk Categories
![Page 11: Lessons Learned from Moodle VLE/LMS Data in the Field](https://reader036.vdocuments.mx/reader036/viewer/2022062306/5a666bdb7f8b9a5d3d8b4b63/html5/thumbnails/11.jpg)
Prediction vs. Final Grade
![Page 12: Lessons Learned from Moodle VLE/LMS Data in the Field](https://reader036.vdocuments.mx/reader036/viewer/2022062306/5a666bdb7f8b9a5d3d8b4b63/html5/thumbnails/12.jpg)
Performance of Predictions
![Page 13: Lessons Learned from Moodle VLE/LMS Data in the Field](https://reader036.vdocuments.mx/reader036/viewer/2022062306/5a666bdb7f8b9a5d3d8b4b63/html5/thumbnails/13.jpg)
Models Change by Week & by Course Type
![Page 14: Lessons Learned from Moodle VLE/LMS Data in the Field](https://reader036.vdocuments.mx/reader036/viewer/2022062306/5a666bdb7f8b9a5d3d8b4b63/html5/thumbnails/14.jpg)
So what?
• Rolling out risk model & X-Ray broadly for teachers
• Providing as useful indicator to augment their decisions (not source of absolute truth)
• Remaining challenge: help teachers interpret probabilistic results
![Page 15: Lessons Learned from Moodle VLE/LMS Data in the Field](https://reader036.vdocuments.mx/reader036/viewer/2022062306/5a666bdb7f8b9a5d3d8b4b63/html5/thumbnails/15.jpg)
Large Scale Research: Student LMS Use vs. Grade
![Page 16: Lessons Learned from Moodle VLE/LMS Data in the Field](https://reader036.vdocuments.mx/reader036/viewer/2022062306/5a666bdb7f8b9a5d3d8b4b63/html5/thumbnails/16.jpg)
Findings: Relationship LMS Time & Grade
• Question: what is the relationship between student time in LMS and their course grade?
• Investigate at student-course level (one student, one course)
• 1.2M students, 34,519 courses, 788 institutions
• Significant, but effect size < 1%
![Page 17: Lessons Learned from Moodle VLE/LMS Data in the Field](https://reader036.vdocuments.mx/reader036/viewer/2022062306/5a666bdb7f8b9a5d3d8b4b63/html5/thumbnails/17.jpg)
But strong effect in some courses (n=7,648, 22%)
![Page 18: Lessons Learned from Moodle VLE/LMS Data in the Field](https://reader036.vdocuments.mx/reader036/viewer/2022062306/5a666bdb7f8b9a5d3d8b4b63/html5/thumbnails/18.jpg)
What makes some for a stronger or weaker relationship?
Tools used? Course design?Quality of activity/effort?
![Page 19: Lessons Learned from Moodle VLE/LMS Data in the Field](https://reader036.vdocuments.mx/reader036/viewer/2022062306/5a666bdb7f8b9a5d3d8b4b63/html5/thumbnails/19.jpg)
Finding: Access to GradesAt every level, probability of higher grade increases with increased use. Causal? Probably not. Good indicator? Absolutely.
![Page 20: Lessons Learned from Moodle VLE/LMS Data in the Field](https://reader036.vdocuments.mx/reader036/viewer/2022062306/5a666bdb7f8b9a5d3d8b4b63/html5/thumbnails/20.jpg)
Finding: Course ContentsMore is not always better. Large jump none to some; then no relationship
![Page 21: Lessons Learned from Moodle VLE/LMS Data in the Field](https://reader036.vdocuments.mx/reader036/viewer/2022062306/5a666bdb7f8b9a5d3d8b4b63/html5/thumbnails/21.jpg)
Finding: Assessments/AssignmentsStudents above mean have lower likelihood of achieving a high grade than students below the mean
![Page 22: Lessons Learned from Moodle VLE/LMS Data in the Field](https://reader036.vdocuments.mx/reader036/viewer/2022062306/5a666bdb7f8b9a5d3d8b4b63/html5/thumbnails/22.jpg)
Finding: Discussion Forums with low/high avg useCompare courses with low forum use to courses with forum use >1 hour / student average
![Page 23: Lessons Learned from Moodle VLE/LMS Data in the Field](https://reader036.vdocuments.mx/reader036/viewer/2022062306/5a666bdb7f8b9a5d3d8b4b63/html5/thumbnails/23.jpg)
Implications
• Move beyond LMS use as proxy for effort (where more is always better), and get at finer-grained learning behaviors that are more useful (e.g. students who are struggling to understand material, students who are not prepared).
• Next Steps
– fine-grained understanding of activity over time (e.g. cramming vs. consistent hard working)
– quality of course materials and course design