measuring engagement: learning analytics in online learning dr. griff richards thompson rivers...
TRANSCRIPT
Measuring Engagement:Learning Analytics in Online Learning
Dr. Griff RichardsThompson Rivers UniversityOpen Learning
Electronic Kazan, 19 April 2011
60oN
49oN
Distance LearningIn Canada
- Is growing, especially in K-12 public schools
-15% of 2008 high school graduates in BC took at least one on-line course
-Increase in “Blended Learning”, online activities for F2F courses.
Kamloops, British Columbia
Learning Engagement
The more a learner interacts with the content and with their peers about the content, the more they are likely to internalize it, and remember it...
Richard Snow, 1980
Interaction Equivalency Theory(Garrison & Anderson)
The quality of the interaction is more important than the source
CourseContent
Instructor
Other learners
Learning Engagement (Kuh)
Learning engagement promotes student retention and academic success.
(George Kuh)
Engagement can be broadly measured for a campus using National Engagement School Survey Instrument (NESSI).
Learning Engagement (Kuh)
Five Factors:1. active and collaborative learning,
2. student-faculty interaction, 3. supportive campus environment, 4. enriching educational experiences, and 5. the level of academic challenge.
Engagement Online
on line
Measuring Engagement• Engagement is “action based on internal
motivation”• Can not directly measure engagement• Must look for external traces, actions that
indicate interest, and involvement
Kuh – NESSI – sample every year of 100,000s of students -> statistical trends, but not individual
as results come too late to correct problems!
Measuring Online Engagement
Engagement is inferred from activity.• Student interaction with Online systems
leaves an “exhaust trail” of their activity• Learning Analytics is not a statistical sample,
it is all the data for all learners• Questions: What patterns to look for? and How to interpret them?
• LMS interaction data e.g. Moodle discussion
• Extract linked list
• Plot interactions in star map
Example: Snapp
SNAPP Visualization
SNAPP shot of Welcome
Students “engaged”
SNAPP shot of Welcome
Individuals with lower engagement, 3 or less messages
Does Activity = Engagement ?
Beer (2010) plotted total LMS interactions with academic grades. Does activity = engagement?
Is Blackboard more engaging than Moodle?
Activity Outside LMS
• As learners become engaged, they move to more engaging channels: email, Elluminate, Skype
• This activity is not tracked by LMS.
• No data available.
Interpretation of Analytics
• Data patterns require verification• Quantitative data requires interpretation--> make and test hypotheses--> create useful models• When we measure something we risk
changing it.– e.g. If learners know we count hits they may
make meaningless hits to fool the system.
Analytics for Learners!
• The same analytics should be able to provide easy to understand information dashboards for students.
How can analytics improve online learning?
• Measuring something is the first step• “Better to measure something than to
measure nothing” (Scrivens)
• Need more data than page hits. We also need to ask learners about their experience, what worked, what needs improvement.
Analytics can help build Quality Systems
Multi-layered system, Timely, Qualitative & Quantitative data
Steady enrolments
Credible learning
Happylearners
Strategic course redesign
Dynamic Evaluation Model (Richards & Devries)
» Preparation Conduct Reflection
• Design
• Facilitation
• Learning
Analytics at the activity level
Dynamic Evaluation Model
» Preparation Conduct Reflection
• Design
• Facilitation
• Learning
Timely feedback enables quick fixes
• Too little
• Too much
• Just right
Masha & Medved
We need to find balance.
Thank you!
Questions?
Measuring Engagement:Learning Analytics in Online Learning
Dr. Griff RichardsThompson Rivers UniversityOpen Learning
Electronic Kazan, 19 April 2011