learning analytics - uts 2013
DESCRIPTION
Learning Analytics: Should we be concerned about the digital “quantification” of learning? University of Technology, Sydney, 10 Dec 2013TRANSCRIPT
![Page 1: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/1.jpg)
Learning Analytics Should we be concerned about the digital “quantification” of learning?
Simon Buckingham Shum Knowledge Media Institute The Open University UK http://simon.buckinghamshum.net http://linkedin.com/in/simon
University of Technology, Sydney, Dec 2013
@sbskmi #LearningAnalytics
![Page 2: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/2.jpg)
aim
leave with a better sense of the analytics design space
+ better questions
than you can ask right now 2
![Page 3: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/3.jpg)
Learning Technology
KMi, OU AI & Argumentation
Learning Dispositions
Human-Centred Informatics
Learning Analytics
Semantic Scientific Publishing
Dialogue / Issue / Argument Mapping
Learning Analytics
+ (Contested) Collective
Intelligence
![Page 4: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/4.jpg)
4
70-strong lab prototyping next generation learning / collaboration / social media
analytics / future internet
![Page 5: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/5.jpg)
5
Funders span disciplines, from basic research to applications:
http://kmi.open.ac.uk/projects/member/simon-buckingham-shum
The Hypermedia Discourse Group Knowledge Media Institute, Open University:
![Page 6: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/6.jpg)
make the invisible visible
make the opaque permeable
make the ephemeral persistent
6
by using digital tools to craft narrative around ideas + documents + multimedia
Scholarship and
Research
![Page 7: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/7.jpg)
make the invisible visible
make the opaque permeable
make the ephemeral persistent
7
by using data about, and generated by learners, to visualize their progress
Learning Analytics
![Page 8: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/8.jpg)
From an analytics product review…
8
![Page 9: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/9.jpg)
From an analytics product review…
“Some have tried to argue that this technology doesn't work out cost effectively when compared to conventional tests... but this misses a huge point. More often than not, we test after the event and discover the problem — but this is too late..”
9
![Page 10: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/10.jpg)
Aquarium Analytics!
10
![Page 11: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/11.jpg)
11
![Page 12: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/12.jpg)
How is your aquatic ecosystem?
“This means that the keeper can be notified before water conditions directly harm the fish—an assured outcome of predictive software that lets you know if it looks like the pH is due to drop, or the temperature is on its way up.
This way, it’s a real fish saver, as opposed to a forensic examiner, post-wipeout.”
(From a review of Seneye, in a hobbyist magazine) 12
![Page 13: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/13.jpg)
How is your learning ecosystem?
This means that the teacher can be notified before learning conditions directly harm the students — an assured outcome of predictive software that lets you know if it looks like engagement is due to drop, or distraction is on its way up.
This way, it’s a real student saver, as opposed to a forensic examiner, post-wipeout.
13
![Page 14: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/14.jpg)
14 L. Johnson, R. Smith, H. Willis, A. Levine, and K. Haywood, The 2011 Horizon Report (Austin, TX: The New Media Consortium,
2011), http://www.nmc.org/pdf/2011-Horizon-Report.pdf
NMC Horizon 2011 Report: Learning Analytics (4-5yrs adoption)
The rise of analytics…
![Page 15: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/15.jpg)
15
Audrey Waters: http://hackeducation.com/2012/11/19/top-ed-tech-trends-of-2012-the-business-of-ed-tech
Ed-Tech startups explosive growth
The rise of analytics…
![Page 16: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/16.jpg)
The rise of analytics…
16
LMS/VLEs + Analytics Publishers + Analytics
![Page 17: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/17.jpg)
17
Back to Aquarium Analytics…
![Page 18: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/18.jpg)
18
fish aquarium science
learners? learning science
instructional design
Back to Aquarium Analytics…
![Page 19: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/19.jpg)
Purdue University Signals: real time traffic-lights for students based on predictive model
19
![Page 20: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/20.jpg)
Purdue University Signals: real time traffic-lights for students based on predictive model
20
Predicted 66%-80% of struggling students who needed help
MODEL: • ACT or SAT score • Overall grade-point average • CMS usage composite • CMS assessment composite • CMS assignment composite • CMS calendar composite
Campbell et al (2007). Academic Analytics: A New Tool for a New Era, EDUCAUSE Review, vol. 42, no. 4 (July/August 2007): 40–57. http://bit.ly/lmxG2x
![Page 21: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/21.jpg)
Purdue University Signals: real time traffic-lights for students based on predictive model
21
“Results thus far show that students who have engaged with
Course Signals have higher average grades and seek out help
resources at a higher rate than other students.”
Pistilli, M. D., Arnold, K. and Bethune, M., Signals: Using Academic Analytics to Promote Student Success. EDUCAUSE Review Online, July/Aug., (2012). http://www.educause.edu/ero/article/signals-using-academic-analytics-promote-student-success
![Page 22: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/22.jpg)
Predictive analytics @open.edu
Registra)on Pa.ern
CRM contact
VLE interac)on
Grades
Demo-‐graphics
? How early can we predict likelihood of dropout, formal withdrawal, failure? Now exploring conventional statistics, machine learning and growing datasets
Library interac)on
OpenLearn interac)on
FutureLearn interac)on
Social App X interac)on OU history
![Page 23: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/23.jpg)
Predictive analytics @open.edu
A.L. Wolff and Z. Zdrahal (2012). Improving Retention by Identifying and Supporting “At-risk” Students. EDUCAUSE Review Online, July-August 2012. http://www.educause.edu/ero/article/improving-retention-identifying-and-supporting-risk-students
Test a range of predictive models:
final result (pass/fail) final numerical score drop in the next TMA score of the next TMA
Demo- graphics
Previous results
VLE activity
Adding in user interaction data from the VLE
![Page 24: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/24.jpg)
Hmmm…
no learning sciences/design underpinning these predictive models of student success
models based on a mix of
institutional know-how about student success, and mining
behavioural data
24
![Page 25: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/25.jpg)
the opportunity for the
learning sciences to combine with your university’s
collective intelligence
25
![Page 26: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/26.jpg)
predictive models are exciting
but there are many other
kinds of analytics
26
![Page 27: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/27.jpg)
Analytics 101 Elaborated version of figure from Doug Clow: h.p://www.slideshare.net/dougclow/the-‐learning-‐analy)cs-‐cycle-‐closing-‐the-‐loop-‐effec)vely (slide 5)
27
What kinds of learners? What kinds of learning?
What data could be generated digitally
from the use context? (you can invent future technologies if need)
Does your theory predict patterns
signifying learning?
What human +/or software
interventions /recommendations?
How to render the analytics, for whom, and will they
understand them?
What analytical tools could be used to find
such patterns?
ethics
![Page 28: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/28.jpg)
Analytics coming to a VLE near you: Blackboard basic summary stats
28
http://www.blackboard.com/Platforms/Analytics/Products/Blackboard-Analytics-for-Learn.aspx
![Page 29: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/29.jpg)
29
Student Activity Dashboard (Erik Duval)
Duval E. (2011) Attention please!: learning analytics for visualization and recommendation. Proceedings of the 1st International Conference on Learning Analytics and Knowledge. Banff, Alberta, Canada: ACM, 9-17.
![Page 30: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/30.jpg)
30
Khan Academy
http://www.youtube.com/watch?v=DLt6mMQH1OY
has extended great instructional movies with a tutoring platform with detailed analytics
![Page 31: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/31.jpg)
31
https://grockit.com/research
Adaptive platforms generate fine-grained analytics on individuals’ curriculum mastery
![Page 32: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/32.jpg)
Intelligent tutoring for skills mastery (CMU) http://oli.cmu.edu
Lovett M, Meyer O and Thille C. (2008) The Open Learning Initiative: Measuring the effectiveness of the OLI statistics course in accelerating student learning. Journal of Interactive Media in Education 14. http://jime.open.ac.uk/article/2008-14/352
“In this study, results showed that OLI-Statistics students [blended learning] learned a full semester’s worth of material in half as much time and performed as well or better than students learning from traditional instruction over a full semester.”
![Page 33: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/33.jpg)
33
Track learner activity with a virtual machine (Abelardo Pardo, LAK13 Conference Keynote)
http://www.slideshare.net/abelardo_pardo/bridging-the-middle-space-with-learning-analytics
![Page 34: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/34.jpg)
34
Track learner activity with a virtual machine (Abelardo Pardo, LAK13 Conference Keynote)
http://www.slideshare.net/abelardo_pardo/bridging-the-middle-space-with-learning-analytics Calvo, R., O’Rourke, S.T., Jones, J., Yacef, K., Reimann, P., 2011. Collaborative Writing Support Tools on the Cloud. IEEE Transactions on Learning Technologies, 4(1):88–97
![Page 35: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/35.jpg)
macro meso micro
analytics 35
![Page 36: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/36.jpg)
Macro/Meso/Micro Learning Analytics
Macro: region/state/national/international
League Tables Data Interoperability Initiatives
![Page 37: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/37.jpg)
Macro/Meso/Micro Learning Analytics
Meso: institution-wide
Macro: region/state/national/international
Univ. Student info Systems Business Intelligence products to improve org processes
![Page 38: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/38.jpg)
Micro: individual user actions
(and hence cohort)
Macro/Meso/Micro Learning Analytics
Meso: institution-wide
Macro: region/state/national/international
Learning Analytics
![Page 39: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/39.jpg)
Micro: individual user actions
(and hence cohort)
Hard distinctions between Learning + Academic analytics may dissolve
Meso: institution-wide
Macro: region/state/national/international
Aggregation of user traces enriches meso + macro analytics with finer-grained process data
…as they get joined up, each level enriches the others
![Page 40: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/40.jpg)
Micro: individual user actions
(and hence cohort)
Hard distinctions between Learning + Academic analytics may dissolve
Meso: institution-wide
Macro: region/state/national/international
Aggregation of user traces enriches meso + macro analytics with finer-grained process data
Breadth + depth from macro + meso levels add power to
micro analytics
…as they get joined up, each level enriches the others
![Page 41: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/41.jpg)
Data Intent
Analytics are not the end, but a means The goal is to optimize the whole system
41
learners
researchers / educators / instructional designers
curriculum theories
pedagogies assessments
tools
desi
gn feedback
intent
outcome
![Page 42: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/42.jpg)
Webinar replay on OU analytics strategy
Belinda Tynan & Simon Buckingham Shum (2013). Designing Systemic Learning Analytics at the Open University. SoLAR Open Course, Strategy & Policy for Systemic Learning Analytics, 11th Oct 2013. http://people.kmi.open.ac.uk/sbs/2013/10/designing-systemic-analytics-at-the-open-university
![Page 43: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/43.jpg)
Optimize the system for what?
43
![Page 44: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/44.jpg)
design analytics to achieve your university’s strategic
goals
(increasingly differentiated as the sector stratifies?)
44
![Page 45: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/45.jpg)
analytics for building learners who handle… • uncertainty
• novel dilemmas • conflicting viewpoints
• VUCA life in C21
45 US military: Volatile/Uncertain/Complex/Ambiguous
?
![Page 46: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/46.jpg)
OECD DeSeCo Final Report Definition & Selection of Key Competencies
46 http://www.deseco.admin.ch
“The OECD has collaborated with a wide range of scholars,
experts and institutions to identify a small set of key competencies that help individuals and whole
societies to meet their goals.”
![Page 47: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/47.jpg)
analytics for social capital
47
![Page 48: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/48.jpg)
Social Network Analysis (SNAPP)
48
What’s going on in these discussion forums?
![Page 49: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/49.jpg)
Social Network Analysis to gain insight into peer-peer and peer-mentor dynamics (SNAPP tool)
49 http://www.slideshare.net/aneeshabakharia/snapp-20minute-presentation Bakharia A and Dawson S. (2011) SNAPP: a bird's-eye view of temporal participant interaction. Proceedings of the 1st International Conference on Learning Analytics and Knowledge. Banff, Alberta, Canada: ACM, 168-173.
![Page 50: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/50.jpg)
Social Network Analysis (SNAPP)
50 http://www.slideshare.net/aneeshabakharia/snapp-20minute-presentation
2 learners connect otherwise separate clusters
tutor only engaging with active students, ignoring disengaged ones on the edge
![Page 51: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/51.jpg)
Social Learning Analytics about to appear in products…
51
http://www.desire2learn.com/products/analytics (this is from a beta demo)
![Page 52: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/52.jpg)
Visualizing activity in OU Facebook sites What do students say about their courses, and what are the patterns of engagement in online communities?
![Page 53: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/53.jpg)
discourse analytics for using language as
a knowledge-building tool
53
![Page 54: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/54.jpg)
1st International Workshop on Discourse-Centric Learning Analytics
analytics that look beneath the surface, and quantify linguistic proxies for ‘deeper learning’
Beyond number / size / frequency of posts; ‘hottest thread’
http://ww
w.glennsasscer.com
/wordpress/w
p-content/uploads/2011/10/iceberg.jpg
solaresearch.org/events/lak/lak13/dcla13
![Page 55: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/55.jpg)
Discourse analytics on webinar textchat
Ferguson, R. and Buckingham Shum, S., Learning analytics to identify exploratory dialogue within synchronous text chat. In: 1st International Conference on Learning Analytics and Knowledge (Banff, Canada, 2011). ACM
Can we spot the quality learning conversations in a 2.5 hr webinar?
![Page 56: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/56.jpg)
-60
-40
-20
0
20
40
60
80
9:2
8 9
:32
9:3
6
9:4
0
9:4
1
9:4
6
9:5
0
9:5
3
9:5
6
10:
00
10:
05
10:
07
10:
07
10:
09
10:
13
10:
17
10:
23
10:
27
10:
31
10:
35
10:
40
10:
45
10:
52
10:
55
11:
04
11:
08
11:
11
11:
17
11:
20
11:
24
11:
26
11:
28
11:
31
11:
32
11:
35
11:
36
11:
38
11:
39
11:
41
11:
44
11:
46
11:
48
11:
52
11:
54
12:
00
12:
03
12:
04
12:
05
Average Exploratory
Discourse analytics on webinar textchat
Sheffield, UK not as sunny as yesterday - still warm Greetings from Hong Kong Morning from Wiltshire, sunny here!
See you! bye for now! bye, and thank you Bye all for now
Given a 2.5 hour webinar, where in the live textchat were the most effective learning conversations? Not at the start and end of a webinar…
Ferguson, R., Wei, Z., He, Y. and Buckingham Shum, S., An Evaluation of Learning Analytics to Identify Exploratory Dialogue in Online Discussions. In: Proc. 3rd International Conference on Learning Analytics & Knowledge (Leuven, BE, 8-12 April, 2013). ACM. http://oro.open.ac.uk/36664
![Page 57: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/57.jpg)
-60
-40
-20
0
20
40
60
80
9:2
8 9
:32
9:3
6
9:4
0
9:4
1
9:4
6
9:5
0
9:5
3
9:5
6
10:
00
10:
05
10:
07
10:
07
10:
09
10:
13
10:
17
10:
23
10:
27
10:
31
10:
35
10:
40
10:
45
10:
52
10:
55
11:
04
11:
08
11:
11
11:
17
11:
20
11:
24
11:
26
11:
28
11:
31
11:
32
11:
35
11:
36
11:
38
11:
39
11:
41
11:
44
11:
46
11:
48
11:
52
11:
54
12:
00
12:
03
12:
04
12:
05
Average Exploratory
Discourse analytics on webinar textchat
Given a 2.5 hour webinar, where in the live textchat were the most effective learning conversations? Not at the start and end of a webinar but if we zoom in on a peak…
Ferguson, R., Wei, Z., He, Y. and Buckingham Shum, S., An Evaluation of Learning Analytics to Identify Exploratory Dialogue in Online Discussions. In: Proc. 3rd International Conference on Learning Analytics & Knowledge (Leuven, BE, 8-12 April, 2013). ACM. http://oro.open.ac.uk/36664
![Page 58: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/58.jpg)
Discourse analytics on webinar textchat
-100
-50
0
50
100
9:2
8 9
:40
9
:50
1
0:00
1
0:07
1
0:17
1
0:31
1
0:45
1
1:04
1
1:17
1
1:26
1
1:32
1
1:38
1
1:44
1
1:52
1
2:03
Averag
Classified as “exploratory
talk”
(more substantive for learning)
“non-exploratory”
Given a 2.5 hour webinar, where in the live textchat were the most effective learning conversations? Not at the start and end of a webinar but if we zoom in on a peak…
Ferguson, R., Wei, Z., He, Y. and Buckingham Shum, S., An Evaluation of Learning Analytics to Identify Exploratory Dialogue in Online Discussions. In: Proc. 3rd International Conference on Learning Analytics & Knowledge (Leuven, BE, 8-12 April, 2013). ACM. http://oro.open.ac.uk/36664
![Page 59: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/59.jpg)
Discourse analytics on webinar textchat
Visualizing by individual user. The gradient of the threshold line is adjusted to every 5 posts in 6 classified as “Exploratory Talk”
Ferguson, R., Wei, Z., He, Y. and Buckingham Shum, S., An Evaluation of Learning Analytics to Identify Exploratory Dialogue in Online Discussions. In: Proc. 3rd International Conference on Learning Analytics & Knowledge (Leuven, BE, 8-12 April, 2013). ACM. http://oro.open.ac.uk/36664
![Page 60: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/60.jpg)
“Rhetorical parsing” to identify constructions signifying scholarly writing
OPEN QUESTION: “… little is known …” “… role … has been elusive” “Current data is insufficient …”
CONTRASTING IDEAS: “… unorthodox view resolves …” “In contrast with previous hypotheses ...” “... inconsistent with past findings ...”
SURPRISE: “We have recently observed ... surprisingly” “We have identified ... unusual” “The recent discovery ... suggests intriguing roles”
http://technologies.kmi.open.ac.uk/cohere/2012/01/09/cohere-plus-automated-rhetorical-annotation De Liddo, A., Sándor, Á. and Buckingham Shum, S., Contested Collective Intelligence: Rationale, Technologies, and a Human-Machine Annotation Study. Computer Supported Cooperative Work, 21, 4-5, (2012), 417-448. http://oro.open.ac.uk/31052 Simsek D, Buckingham Shum S, Sándor Á, De Liddo A and Ferguson R. (2013) XIP Dashboard: http://oro.open.ac.uk/37391
![Page 61: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/61.jpg)
“What are the key contributions of this text?
http://technologies.kmi.open.ac.uk/cohere/2012/01/09/cohere-plus-automated-rhetorical-annotation De Liddo, A., Sándor, Á. and Buckingham Shum, S., Contested Collective Intelligence: Rationale, Technologies, and a Human-Machine Annotation Study. Computer Supported Cooperative Work, 21, 4-5, (2012), 417-448. http://oro.open.ac.uk/31052 Simsek D, Buckingham Shum S, Sándor Á, De Liddo A and Ferguson R. (2013) XIP Dashboard: http://oro.open.ac.uk/37391
Human analyst Computational analyst
![Page 62: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/62.jpg)
Social Learning Analytics
Buckingham Shum, Simon and Ferguson, Rebecca (2012). Social Learning Analytics. Journal of Educational Technology and Society, 15(3) pp. 3–26. http://oro.open.ac.uk/34092
• Explosive growth in social media
• The open/free content paradigm
• Evidence of a global shift in societal attitudes which increasingly values participation
• Innovation depends on reciprocal social relationships, tacit knowing
![Page 63: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/63.jpg)
intrinsic motivation self-regulation
resilience
63
![Page 64: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/64.jpg)
Why do dispositions matter?
64
“Knowledge of methods alone will not suffice: there must be the desire, the will, to employ them. This desire is an affair of personal disposition.”
John Dewey
Dewey, J. How We Think: A Restatement of the Relation of Reflective Thinking to the Educative Process. Heath and Co, Boston, 1933
![Page 65: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/65.jpg)
“In the growth mindset, people believe that their talents and abilities can be developed through passion, education, and persistence …
It’s about a commitment to … taking informed risks … surrounding yourself with people who will challenge you to grow”
Carol Dweck
65 Interview with Carol Dweck: http://interviewscoertvisser.blogspot.co.uk/2007/11/interview-with-carol-dweck_4897.html
Why do dispositions matter?
![Page 66: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/66.jpg)
“We’re looking at the profiles of what it means to be effective in the 21st century. […] Resilience will be the defining concept. When challenged and bent, you learn and bounce back stronger.”
“Dispositions are now at least as important as Knowledge and Skills. …They cannot be taught. They can only be cultivated.”
John Seely Brown
66
US Dept. of Educ. http://reimaginingeducation.org conference (May 28, 2013) Dispositions clip: http://www.c-spanvideo.org/clip/4457327 Whole talk: http://www.c-spanvideo.org/program/SecD
Why do dispositions matter?
![Page 67: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/67.jpg)
“It’s more than knowledge and skills.
For the innovation economy, dispositions come into play: readiness to collaborate; attention to multiple perspectives; initiative; persistence; curiosity. The purpose of learning in the 21st century is not to recite inert knowledge but to transform it. It’s time to change the subject.”
67
LearningREimagined project: http://learning-reimagined.com Larry Rosenstock: http://audioboo.fm/boos/1669375-50-seconds-of-larry-rosenstock-ceo-of-hightechhigh-on-how-he-would-re-imagine-learning
Larry Rosenstock
Why do dispositions matter?
![Page 68: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/68.jpg)
How can we model and quantify learning
dispositions in order to develop analytics?
68
![Page 69: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/69.jpg)
Dispositional Learning Analytics Workshop
69 http://learningemergence.net/events/lasi-dla-wkshp
![Page 70: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/70.jpg)
Validated as loading onto 7 dimensions of “Learning Power”
Changing & Learning
Meaning Making
Critical Curiosity
Creativity
Learning Relationships
Strategic Awareness
Resilience
Being Stuck & Static
Data Accumulation
Passivity
Being Rule Bound
Isolation & Dependence
Being Robotic
Fragility & Dependence
Ruth Deakin Crick Grad. School of Education
![Page 71: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/71.jpg)
Analytics for lifelong/lifewide learning dispositions: ELLI
Buckingham Shum, S. and Deakin Crick, R. (2012). Learning Dispositions and Transferable Competencies: Pedagogy, Modelling and Learning Analytics. Proc. 2nd Int. Conf. Learning Analytics & Knowledge. (29 Apr-2 May, Vancouver). Eprint: http://oro.open.ac.uk/32823
![Page 72: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/72.jpg)
Learning to Learn: 7 Dimensions of Learning Power Factor analysis of the literature plus expert interviews: identified seven dimensions of effective “learning power”, since validated empirically with learners at many levels. (Deakin Crick, Broadfoot and Claxton, 2004)
![Page 73: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/73.jpg)
Learning to Learn: 7 Dimensions of Learning Power
73
![Page 74: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/74.jpg)
next step: platforms for Dispositional Learning
Analytics
74 DLA Workshop: http://learningemergence.net/events/lasi-dla-wkshp
![Page 75: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/75.jpg)
Primary School EnquiryBloggers Bushfield School, Wolverton, UK
EnquiryBlogger: blogging for Learning Power & Authentic Enquiry http://learningemergence.net/2012/06/20/enquiryblogger-for-learning-power-authentic-enquiry
![Page 76: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/76.jpg)
Masters level EnquiryBloggers Graduate School of Education, University of Bristol
EnquiryBlogger: blogging for Learning Power & Authentic Enquiry http://learningemergence.net/2012/06/20/enquiryblogger-for-learning-power-authentic-enquiry
![Page 77: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/77.jpg)
EnquiryBlogger dashboard – direct
navigation to learner’s blogs from the visual
analytic
![Page 78: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/78.jpg)
Could a platform generate an ELLI profile from user traces?
Shaofu Huang: Prototyping Learning Power Modelling in SocialLearn http://www.open.ac.uk/blogs/SocialLearnResearch/2012/06/20/social-learning-analytics-symposium
Different social network patterns
in different contexts may
load onto Learning
Relationships
Questioning and challenging may load onto Critical
Curiosity
Sharing relevant resources from other contexts may load onto
Meaning Making
Repeated attempts to pass
an online test may load onto
Resilience
![Page 79: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/79.jpg)
Your most recent mood comment: “Great, at last I have found all the resources that I have been looking for, thanks to!Steve and Ellen.!
In your last discussion with your mentor, you decided to work on your resilience by taking on more learning challenges
Your ELLI Spider shows that you have made a start on working on your resilience, and that you are also beginning to work on your creativity, which you identified as another area to work on.
1 2 3
45
Envisioning a social learning analytics dashboard
Ferguson R and Buckingham Shum S. (2012) Social Learning Analytics: Five Approaches. Proc. 2nd International Conference on Learning Analytics & Knowledge. Vancouver, 29 Apr-2 May: ACM: New York, 23-33. DOI: http://dx.doi.org/10.1145/2330601.2330616 Eprint: http://oro.open.ac.uk/32910
![Page 80: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/80.jpg)
towards wholistic learning analytics
Ruth Deakin Crick, Howard Green, Steven Barr
![Page 81: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/81.jpg)
Towards wholistic analytics on the health of a learning community
81
Academic results are important but less easily quantifiable measures of success are vital to
many institutions’ vision to nurture
• life-long/life-wide learners • employability skills • citizenship
• self-confidence • teamwork • emotional wellbeing…
![Page 82: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/82.jpg)
Hierarchical Process Modelling (Univ. Bristol PeriMeta tool)
82
Seeing a learning community as a complex adaptive system requires the voices of learners, teachers,
leaders, parents and ‘external’ stakeholders
![Page 83: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/83.jpg)
Hierarchical Process Modelling (Univ. Bristol PeriMeta tool)
83
The core mission and values of a network of school academies
![Page 84: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/84.jpg)
Hierarchical Process Modelling (Univ. Bristol PeriMeta tool)
84
Hierarchical Process Modelling transforms
qualitative and quantitative inputs into a multi-level visual
analytic
![Page 85: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/85.jpg)
‘Italian Flag’ visual analytic
85
Degree of green / white / red reflects current certainty over availability of evidence supporting / unknown / challenging
![Page 86: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/86.jpg)
‘Italian Flag’ model
86
Converting learner/leader/teachers’ linguistic survey ratings and confidence into quantitative values
![Page 87: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/87.jpg)
‘Italian Flag’ visual analytic
87
Degree of green / white / red reflects current certainty over availability of evidence supporting / unknown / challenging
![Page 88: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/88.jpg)
thorny issues
88
![Page 89: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/89.jpg)
learning analytics
are not neutral
89
![Page 90: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/90.jpg)
Accounting tools are not neutral
“accounting tools...do not simply aid the measurement of economic activity, they shape the reality they measure”
Du Gay, P. and Pryke, M. (2002) Cultural Economy: Cultural Analysis and Commercial Life. Sage, London. pp. 12-13
![Page 91: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/91.jpg)
cf. Bowker and Starr’s “Sorting Things Out” on classification schemes
Buckingham Shum, S. and Deakin Crick, R. (2012). Learning Dispositions and Transferable Competencies: Pedagogy, Modelling and Learning Analytics. Proc. 2nd Int. Conf. Learning Analytics & Knowledge. (29 Apr-2 May, 2012, Vancouver, BC). ACM. Eprint: http://oro.open.ac.uk/32823
“A marker of the health of the learning analytics field will be the quality of debate around what the technology renders visible and leaves invisible.”
![Page 92: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/92.jpg)
Analytics are infused with human values Data does not ‘speak for itself’
92
What kinds of learners? What kinds of learning?
What data could be generated digitally
from the use context? (you can invent future technologies if need)
Does your theory predict patterns
signifying learning?
What human +/or software
interventions /recommendations?
How to render the analytics, for whom, and will they
understand them?
What analytical tools could be used to find
such patterns?
ethics
![Page 93: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/93.jpg)
The Wal-Martification of education?
93 http://chronicle.com/blogs/techtherapy/2012/05/02/episode-95-learning-analytics-could-lead-to-wal-martification-of-college http://lak12.wikispaces.com/Recordings
“The basic question is not what can we measure? The basic question is
what does a good education look like?
Big questions.
“data narrowness” “instrumental learning”
“students with no curiosity”
![Page 94: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/94.jpg)
94
Will staff know how to read and write analytics?
This will become a key literacy.
![Page 95: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/95.jpg)
Learning technology is not neutral
95
Any technology embodies epistemological, pedagogical and
assessment assumptions
epistemology
pedagogy assessment
Knight S, Buckingham Shum S and Littleton K. (2013) Epistemology, Pedagogy, Assessment and Learning Analytics. Proc. 3rd International Conference on Learning Analytics & Knowledge. Leuven, BE: ACM, 75-84 Open Access Eprint: http://oro.open.ac.uk/36635
![Page 96: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/96.jpg)
Algorithms are not neutral
96
Algorithms increasingly shape our lives: we need to go in eyes wide open, and encourage critical debate
http://governingalgorithms.org
![Page 97: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/97.jpg)
join the learning analytics global
community 97
![Page 98: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/98.jpg)
Join the community…
98
http://SoLAResearch.org
http://LAKconference.org
replays of all previous
conference presentations
![Page 99: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/99.jpg)
Join the community…
99 http://www.solaresearch.org/events/lasi
replays of all sessions
![Page 100: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/100.jpg)
JISC Briefings on Learning Analytics
100 http://publications.cetis.ac.uk/c/analytics
![Page 101: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/101.jpg)
EDUCAUSE Briefings on Learning Analytics
101 http://www.educause.edu/library/learning-analytics
![Page 102: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/102.jpg)
Learning Analytics Policy Brief (UNESCO • IITE)
102 http://bit.ly/LearningAnalytics
![Page 103: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/103.jpg)
Systems leadership and learning: LearningEmergence.net
![Page 104: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/104.jpg)
what this all means
104
![Page 105: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/105.jpg)
105
Academic Culture
data-intensive learning sciences/
educ research
C21 Qualities
place these on a firm empirical evidence
base
Practitioner Culture
evidence impact of learning designs;
timely interventions
The big shifts that analytics could bring…
Organisational Culture
evidence-based decisions and org learning
![Page 106: Learning Analytics - UTS 2013](https://reader033.vdocuments.mx/reader033/viewer/2022042623/54c687524a7959a2128b4629/html5/thumbnails/106.jpg)
106
sensemaking
meets computation creative intelligence +
computational thinking
pedagogical innovation
how do learning analytics change
student experience?
educator data literacy
how do staff learn to read and write
analytics?
The new research+practice vista…
data-culture dynamics
how do HEIs manage the embedding of real time analytics
services?