LEARNING ANALYTICS FOR C21 DISPOSITIONS & SKILLS
Simon Buckingham Shum Knowledge Media Institute, Open U. UK
http://simon.buckinghamshum.net @sbskmi
http://net.educause.edu/eliweb121
L.A. FRAMEWORK TO THINK WITH…
Discipline knowledge
Educator owns and manages a single dataset
2
L.A. FRAMEWORK TO THINK WITH…
Discipline knowledge
Educator owns and manages a single dataset
Educator owns and manages mul2ple datasets
3
L.A. FRAMEWORK TO THINK WITH…
Discipline knowledge
Educator owns and manages a single dataset
Educator owns and manages mul2ple datasets
Learners add their own datasets
4
L.A. FRAMEWORK TO THINK WITH…
Discipline knowledge
Educator owns and manages a single dataset
Educator owns and manages mul2ple datasets
Learners add their own datasets
Hybrid closed + open datasets
5
L.A. FRAMEWORK TO THINK WITH…
Discipline knowledge
Educator owns and manages a single dataset
Educator owns and manages mul2ple datasets
Learners add their own datasets
Hybrid closed + open datasets
Hybrid closed + open analy2cs
6
L.A. FRAMEWORK TO THINK WITH…
Discipline knowledge
Educator owns and manages a single dataset
Educator owns and manages mul2ple datasets
Learners add their own datasets
Hybrid closed + open datasets
Hybrid closed + open analy2cs
Focus of most LA effort
beginning to move
towards these more complex spaces
7
L.A. FRAMEWORK TO THINK WITH…
Discipline knowledge
Educator owns and manages a single dataset
Educator owns and manages mul2ple datasets
Learners add their own datasets
Hybrid closed + open datasets
Hybrid closed + open analy2cs
Focus of most LA effort
beginning to move
towards these more complex spaces
http://solaresearch.org/OpenLearningAnalytics.pdf
8
L.A. FRAMEWORK TO THINK WITH…
Discipline knowledge C21 Learning Capaci5es
Educator owns and manages a single dataset
Educator owns and manages mul2ple datasets
Learners add their own datasets
Hybrid closed + open datasets
Hybrid closed + open analy2cs
cri5cal for learner
engagement, and authen5c
learning
Focus of most LA effort
beginning to move
towards these more complex spaces
9
“We are preparing students for jobs that do not exist yet, that will use technologies that have not been invented yet, in order to solve problems that are not even problems yet.”
“Shift Happens” http://shifthappens.wikispaces.com
LEARNING ANALYTICS FOR THIS?
10
LEARNING ANALYTICS FOR THIS?
“The test of successful education is not the amount of knowledge that pupils take away from school, but their appetite to know and their capacity to learn.”
Sir Richard Livingstone, 1941
11
ANALYTICS FOR… C21 SKILLS?
LEARNING HOW TO LEARN? AUTHENTIC ENQUIRY?
social capital critical questioning argumentation citizenship habits of mind resilience
collaboration creativity metacognition identity readiness sensemaking
engagement motivation emotional intelligence
12 12
L.A. FRAMEWORK TO THINK WITH…
Discipline knowledge C21 Learning Capaci5es
Educator owns and manages a single dataset
Educator owns and manages mul2ple datasets
Learners add their own datasets
Hybrid closed + open datasets
Hybrid closed + open analy2cs
More LA effort needed
e.g.
1. Disposition Analytics
2. Discourse Analytics
Focus of most LA effort
beginning to move
towards these more complex spaces
13
Analytics for learning dispositions
14
ELLI: EFFECTIVE LIFELONG LEARNING INVENTORY WEB QUESTIONNAIRE 72 ITEMS (CHILDREN AND ADULT VERSIONS: USED IN SCHOOLS, UNIVERSITIES AND WORKPLACE)
15
Buckingham Shum, S. and Deakin Crick, R. (2012). Learning Disposi5ons and Transferable Competencies: Pedagogy, Modelling, and Learning Analy5cs. Accepted to 2nd Interna*onal Conference on Learning Analy*cs & Knowledge (Vancouver, 29 Apr – 2 May, 2012).
VALIDATED AS LOADING ONTO 7 DIMENSIONS OF “LEARNING POWER”
Changing & Learning
Meaning Making
Critical Curiosity
Creativity
Learning Relationships
Strategic Awareness
Resilience
Being Stuck & Static
Data Accumulation
Passivity
Being Rule Bound
Isolation & Dependence
Being Robotic
Fragility & Dependence
16
ELLI GENERATES A 7-DIMENSIONAL SPIDER DIAGRAM OF HOW THE LEARNER SEES THEMSELF
17
Bristol and Open University are now embedding ELLI in learning software.
Basis for a mentored-‐discussion on how the learner
sees him/herself, and strategies for strengthening
the profile
17
ADDING IMAGERY TO ELLI DIMENSIONS TO CONNECT WITH LEARNER IDENTITY
18
ELLI GENERATES COHORT DATA FOR EACH DIMENSION
19
…DRILLING DOWN ON A SPECIFIC DIMENSION
20
Plugin visualizes blog categories,
mirroring the ELLI spider
ENQUIRYBLOGGER: TUNING WORDPRESS AS AN ELLI-BASED LEARNING JOURNAL
21
Standard Wordpress editor
Categories from ELLI
21
22
ENQUIRYBLOGGER: COHORT DASHBOARD
LEARNINGEMERGENCE.NET more on analytics for learning to learn and authentic enquiry
23
Analytics for learning
conversations
24
DISCOURSE LEARNING ANALYTICS
Effective learning conversations display some typical characteristics which learners can and
should be helped to master
Learners’ written, online conversations can be analysed computationally for patterns signifying
weaker and stronger forms of contribution
25
SOCIO-CULTURAL DISCOURSE ANALYSIS (MERCER ET AL, OU)
• Disputational talk, characterised by disagreement and individualised decision making.
• Cumulative talk, in which speakers build positively but uncritically on what the others have said.
• Exploratory talk, in which partners engage critically but constructively with each other's ideas.
Mercer, N. (2004). Sociocultural discourse analysis: analysing classroom talk as a social mode of thinking. Journal of Applied Linguistics, 1(2), 137-168.
26
• Exploratory talk, in which partners engage critically but constructively with each other's ideas. • Statements and suggestions are offered for joint consideration.
• These may be challenged and counter-challenged, but challenges are justified and alternative hypotheses are offered.
• Partners all actively participate and opinions are sought and considered before decisions are jointly made.
• Compared with the other two types, in Exploratory talk knowledge is made more publicly accountable and reasoning is more visible in the talk.
Mercer, N. (2004). Sociocultural discourse analysis: analysing classroom talk as a social mode of thinking. Journal of Applied Linguistics, 1(2), 137-168.
SOCIO-CULTURAL DISCOURSE ANALYSIS (MERCER ET AL, OU)
27
ANALYTICS FOR IDENTIFYING EXPLORATORY TALK
Elluminate sessions can be very long – lasting for hours or even covering days of a conference
It would be useful if we could identify where quality learning conversations seem to be taking place, so we can recommend those sessions, and not have to sit through online chat about virtual biscuits
Ferguson, R. and Buckingham Shum, S. Learning analytics to identify exploratory dialogue within synchronous text chat. 1st International Conference on Learning Analytics & Knowledge (Banff, Canada, 27 Mar-1 Apr, 2011)
28
De Liddo, A., Buckingham Shum, S., Quinto, I., Bachler, M. and Cannavacciuolo, L. Discourse-centric learning analytics. 1st International Conference on Learning Analytics & Knowledge (Banff, 27 Mar-1 Apr, 2011)
KMI’S COHERE: A WEB DELIBERATION PLATFORM ENABLING SEMANTIC SOCIAL NETWORK AND DISCOURSE NETWORK ANALYTICS
Rebecca is playing the role of broker,
connecting 2 peers’ contributions in
meaningful ways
29
DISCOURSE ANALYSIS
BACKGROUND KNOWLEDGE:
Recent studies indicate …
… the previously proposed …
… is universally accepted ...
NOVELTY:
... new insights provide direct evidence ...
... we suggest a new ... approach ...
... results define a novel role ...
OPEN QUESTION: … li[le is known …
… role … has been elusive
Current data is insufficient …
GENERALIZING: ... emerging as a promising approach
Our understanding ... has grown exponen2ally ...
... growing recogni2on of the
importance ...
CONRASTING IDEAS: … unorthodox view resolves … paradoxes …
In contrast with previous hypotheses ...
... inconsistent with past findings ...
SIGNIFICANCE: studies ... have provided important advances
Knowledge ... is crucial for ... understanding
valuable informa2on ... from studies
SURPRISE: We have recently observed ... surprisingly
We have iden2fied ... unusual
The recent discovery ... suggests intriguing roles
SUMMARIZING: The goal of this study ...
Here, we show ...
Altogether, our results ... indicate
Xerox’s parser can detect the presence of ‘knowledge-‐level’ moves in text:
Ágnes Sándor & OLnet Project: http://olnet.org/node/512
De Liddo, A., Sándor, Á. and Buckingham Shum, S. (In Press). Contested Collective Intelligence: Rationale, Technologies, and a Human-Machine Annotation Study. Computer Supported Cooperative Work Journal
30
NEXT STEPS
SOCIAL LEARNING ANALYTICS: Develop this framework to integrate social, discourse, disposition and other process-centric analytics
DISPOSITION ANALYTICS: Extend the capabilities of the ELLI ‘learning power’ platform using real-time analytics data from online learner activity DISCOURSE ANALYTICS: human+machine annotation of written discourse and argument maps
31
IN MORE DETAIL… Social Learning Analy5cs § Buckingham Shum, S. and Ferguson, R. (2011). Social Learning Analy5cs. Available as: Technical Report KMI-‐11-‐01, Knowledge
Media Ins2tute, The Open University, UK. h[p://kmi.open.ac.uk/publica2ons/techreport/kmi-‐11-‐01
Discourse Analy5cs § De Liddo, A., Buckingham Shum, S., Quinto, I., Bachler, M. and Cannavacciuolo, L. (2011). Discourse-‐Centric Learning Analy5cs.
1st Interna*onal Conference on Learning Analy*cs & Knowledge (Banff, 27 Mar-‐1 Apr, 2011). Eprint: h[p://oro.open.ac.uk/25829 § Ferguson, R. and Buckingham Shum, S. (2011). Learning Analy5cs to Iden5fy Exploratory Dialogue Within Synchronous Text
Chat. 1st Interna*onal Conference on Learning Analy*cs & Knowledge (Banff, Canada, 27 Mar-‐1 Apr, 2011). Eprint: h[p://oro.open.ac.uk/28955
§ De Liddo, A., Sándor, Á. and Buckingham Shum, S. (2012, In Press). Contested Collec5ve Intelligence: Ra5onale, Technologies, and a Human-‐Machine Annota5on Study. Computer Supported Coopera*ve Work. DOI: 10.1007/s10606-‐011-‐9155-‐x. h[p://www.springerlink.com/content/23n1408l9g06v062
Disposi5on Analy5cs § Ferguson, R., Buckingham Shum, S. and Deakin Crick, R. (2011). EnquiryBlogger: Using Widgets to Support Awareness and
Reflec5on in a PLE SeQng. 1st Workshop on Awareness and Reflec*on in Personal Learning Environments, PLE Conference 2011, 11-‐13 July 2011, Southampton, UK. Eprint: h[p://oro.open.ac.uk/30598
§ Buckingham Shum, S. and Deakin Crick, R (2012). Learning Disposi5ons and Transferable Competencies: Pedagogy, Modelling, and Learning Analy5cs. Accepted to 2nd Interna*onal Conference on Learning Analy*cs & Knowledge (Vancouver, 29 Apr – 2 May, 2012). Working drap under revision: h[p://projects.kmi.open.ac.uk/hyperdiscourse/docs/SBS-‐RDC-‐review.pdf
SUMMARY
Discipline knowledge C21 Learning Capaci5es
Educator owns and manages a single dataset
Educator owns and manages mul2ple datasets
Learners add their own datasets
Hybrid closed + open datasets
Hybrid closed + open analy2cs
More LA effort needed
We need analytics tuned to generic capacities which equip learners for novel challenges
Focus of most LA effort
mastery of core knowledge and
skills in training is vital, but no longer
sufficient
33