![Page 1: Measuring Engagement: Learning Analytics in Online Learning Dr. Griff Richards Thompson Rivers University Open Learning Electronic Kazan, 19 April 2011](https://reader035.vdocuments.mx/reader035/viewer/2022062517/56649eb05503460f94bb5e82/html5/thumbnails/1.jpg)
Measuring Engagement:Learning Analytics in Online Learning
Dr. Griff RichardsThompson Rivers UniversityOpen Learning
Electronic Kazan, 19 April 2011
![Page 2: Measuring Engagement: Learning Analytics in Online Learning Dr. Griff Richards Thompson Rivers University Open Learning Electronic Kazan, 19 April 2011](https://reader035.vdocuments.mx/reader035/viewer/2022062517/56649eb05503460f94bb5e82/html5/thumbnails/2.jpg)
60oN
49oN
Distance LearningIn Canada
- Is growing, especially in K-12 public schools
-15% of 2008 high school graduates in BC took at least one on-line course
-Increase in “Blended Learning”, online activities for F2F courses.
Kamloops, British Columbia
![Page 3: Measuring Engagement: Learning Analytics in Online Learning Dr. Griff Richards Thompson Rivers University Open Learning Electronic Kazan, 19 April 2011](https://reader035.vdocuments.mx/reader035/viewer/2022062517/56649eb05503460f94bb5e82/html5/thumbnails/3.jpg)
Learning Engagement
The more a learner interacts with the content and with their peers about the content, the more they are likely to internalize it, and remember it...
Richard Snow, 1980
![Page 4: Measuring Engagement: Learning Analytics in Online Learning Dr. Griff Richards Thompson Rivers University Open Learning Electronic Kazan, 19 April 2011](https://reader035.vdocuments.mx/reader035/viewer/2022062517/56649eb05503460f94bb5e82/html5/thumbnails/4.jpg)
Interaction Equivalency Theory(Garrison & Anderson)
The quality of the interaction is more important than the source
CourseContent
Instructor
Other learners
![Page 5: Measuring Engagement: Learning Analytics in Online Learning Dr. Griff Richards Thompson Rivers University Open Learning Electronic Kazan, 19 April 2011](https://reader035.vdocuments.mx/reader035/viewer/2022062517/56649eb05503460f94bb5e82/html5/thumbnails/5.jpg)
Learning Engagement (Kuh)
Learning engagement promotes student retention and academic success.
(George Kuh)
Engagement can be broadly measured for a campus using National Engagement School Survey Instrument (NESSI).
![Page 6: Measuring Engagement: Learning Analytics in Online Learning Dr. Griff Richards Thompson Rivers University Open Learning Electronic Kazan, 19 April 2011](https://reader035.vdocuments.mx/reader035/viewer/2022062517/56649eb05503460f94bb5e82/html5/thumbnails/6.jpg)
Learning Engagement (Kuh)
Five Factors:1. active and collaborative learning,
2. student-faculty interaction, 3. supportive campus environment, 4. enriching educational experiences, and 5. the level of academic challenge.
![Page 7: Measuring Engagement: Learning Analytics in Online Learning Dr. Griff Richards Thompson Rivers University Open Learning Electronic Kazan, 19 April 2011](https://reader035.vdocuments.mx/reader035/viewer/2022062517/56649eb05503460f94bb5e82/html5/thumbnails/7.jpg)
Engagement Online
on line
![Page 8: Measuring Engagement: Learning Analytics in Online Learning Dr. Griff Richards Thompson Rivers University Open Learning Electronic Kazan, 19 April 2011](https://reader035.vdocuments.mx/reader035/viewer/2022062517/56649eb05503460f94bb5e82/html5/thumbnails/8.jpg)
Measuring Engagement• Engagement is “action based on internal
motivation”• Can not directly measure engagement• Must look for external traces, actions that
indicate interest, and involvement
Kuh – NESSI – sample every year of 100,000s of students -> statistical trends, but not individual
as results come too late to correct problems!
![Page 9: Measuring Engagement: Learning Analytics in Online Learning Dr. Griff Richards Thompson Rivers University Open Learning Electronic Kazan, 19 April 2011](https://reader035.vdocuments.mx/reader035/viewer/2022062517/56649eb05503460f94bb5e82/html5/thumbnails/9.jpg)
Measuring Online Engagement
Engagement is inferred from activity.• Student interaction with Online systems
leaves an “exhaust trail” of their activity• Learning Analytics is not a statistical sample,
it is all the data for all learners• Questions: What patterns to look for? and How to interpret them?
![Page 10: Measuring Engagement: Learning Analytics in Online Learning Dr. Griff Richards Thompson Rivers University Open Learning Electronic Kazan, 19 April 2011](https://reader035.vdocuments.mx/reader035/viewer/2022062517/56649eb05503460f94bb5e82/html5/thumbnails/10.jpg)
• LMS interaction data e.g. Moodle discussion
• Extract linked list
• Plot interactions in star map
Example: Snapp
![Page 11: Measuring Engagement: Learning Analytics in Online Learning Dr. Griff Richards Thompson Rivers University Open Learning Electronic Kazan, 19 April 2011](https://reader035.vdocuments.mx/reader035/viewer/2022062517/56649eb05503460f94bb5e82/html5/thumbnails/11.jpg)
SNAPP Visualization
![Page 12: Measuring Engagement: Learning Analytics in Online Learning Dr. Griff Richards Thompson Rivers University Open Learning Electronic Kazan, 19 April 2011](https://reader035.vdocuments.mx/reader035/viewer/2022062517/56649eb05503460f94bb5e82/html5/thumbnails/12.jpg)
SNAPP shot of Welcome
Students “engaged”
![Page 13: Measuring Engagement: Learning Analytics in Online Learning Dr. Griff Richards Thompson Rivers University Open Learning Electronic Kazan, 19 April 2011](https://reader035.vdocuments.mx/reader035/viewer/2022062517/56649eb05503460f94bb5e82/html5/thumbnails/13.jpg)
SNAPP shot of Welcome
Individuals with lower engagement, 3 or less messages
![Page 14: Measuring Engagement: Learning Analytics in Online Learning Dr. Griff Richards Thompson Rivers University Open Learning Electronic Kazan, 19 April 2011](https://reader035.vdocuments.mx/reader035/viewer/2022062517/56649eb05503460f94bb5e82/html5/thumbnails/14.jpg)
Does Activity = Engagement ?
Beer (2010) plotted total LMS interactions with academic grades. Does activity = engagement?
Is Blackboard more engaging than Moodle?
![Page 15: Measuring Engagement: Learning Analytics in Online Learning Dr. Griff Richards Thompson Rivers University Open Learning Electronic Kazan, 19 April 2011](https://reader035.vdocuments.mx/reader035/viewer/2022062517/56649eb05503460f94bb5e82/html5/thumbnails/15.jpg)
Activity Outside LMS
• As learners become engaged, they move to more engaging channels: email, Elluminate, Skype
• This activity is not tracked by LMS.
• No data available.
![Page 16: Measuring Engagement: Learning Analytics in Online Learning Dr. Griff Richards Thompson Rivers University Open Learning Electronic Kazan, 19 April 2011](https://reader035.vdocuments.mx/reader035/viewer/2022062517/56649eb05503460f94bb5e82/html5/thumbnails/16.jpg)
Interpretation of Analytics
• Data patterns require verification• Quantitative data requires interpretation--> make and test hypotheses--> create useful models• When we measure something we risk
changing it.– e.g. If learners know we count hits they may
make meaningless hits to fool the system.
![Page 17: Measuring Engagement: Learning Analytics in Online Learning Dr. Griff Richards Thompson Rivers University Open Learning Electronic Kazan, 19 April 2011](https://reader035.vdocuments.mx/reader035/viewer/2022062517/56649eb05503460f94bb5e82/html5/thumbnails/17.jpg)
Analytics for Learners!
• The same analytics should be able to provide easy to understand information dashboards for students.
![Page 18: Measuring Engagement: Learning Analytics in Online Learning Dr. Griff Richards Thompson Rivers University Open Learning Electronic Kazan, 19 April 2011](https://reader035.vdocuments.mx/reader035/viewer/2022062517/56649eb05503460f94bb5e82/html5/thumbnails/18.jpg)
How can analytics improve online learning?
• Measuring something is the first step• “Better to measure something than to
measure nothing” (Scrivens)
• Need more data than page hits. We also need to ask learners about their experience, what worked, what needs improvement.
![Page 19: Measuring Engagement: Learning Analytics in Online Learning Dr. Griff Richards Thompson Rivers University Open Learning Electronic Kazan, 19 April 2011](https://reader035.vdocuments.mx/reader035/viewer/2022062517/56649eb05503460f94bb5e82/html5/thumbnails/19.jpg)
Analytics can help build Quality Systems
Multi-layered system, Timely, Qualitative & Quantitative data
Steady enrolments
Credible learning
Happylearners
Strategic course redesign
![Page 20: Measuring Engagement: Learning Analytics in Online Learning Dr. Griff Richards Thompson Rivers University Open Learning Electronic Kazan, 19 April 2011](https://reader035.vdocuments.mx/reader035/viewer/2022062517/56649eb05503460f94bb5e82/html5/thumbnails/20.jpg)
Dynamic Evaluation Model (Richards & Devries)
» Preparation Conduct Reflection
• Design
• Facilitation
• Learning
Analytics at the activity level
![Page 21: Measuring Engagement: Learning Analytics in Online Learning Dr. Griff Richards Thompson Rivers University Open Learning Electronic Kazan, 19 April 2011](https://reader035.vdocuments.mx/reader035/viewer/2022062517/56649eb05503460f94bb5e82/html5/thumbnails/21.jpg)
Dynamic Evaluation Model
» Preparation Conduct Reflection
• Design
• Facilitation
• Learning
Timely feedback enables quick fixes
![Page 22: Measuring Engagement: Learning Analytics in Online Learning Dr. Griff Richards Thompson Rivers University Open Learning Electronic Kazan, 19 April 2011](https://reader035.vdocuments.mx/reader035/viewer/2022062517/56649eb05503460f94bb5e82/html5/thumbnails/22.jpg)
• Too little
• Too much
• Just right
Masha & Medved
We need to find balance.
![Page 23: Measuring Engagement: Learning Analytics in Online Learning Dr. Griff Richards Thompson Rivers University Open Learning Electronic Kazan, 19 April 2011](https://reader035.vdocuments.mx/reader035/viewer/2022062517/56649eb05503460f94bb5e82/html5/thumbnails/23.jpg)
Thank you!
Questions?
![Page 24: Measuring Engagement: Learning Analytics in Online Learning Dr. Griff Richards Thompson Rivers University Open Learning Electronic Kazan, 19 April 2011](https://reader035.vdocuments.mx/reader035/viewer/2022062517/56649eb05503460f94bb5e82/html5/thumbnails/24.jpg)
Measuring Engagement:Learning Analytics in Online Learning
Dr. Griff RichardsThompson Rivers UniversityOpen Learning
Electronic Kazan, 19 April 2011