requirements for learning analytics
TRANSCRIPT
Requirements for Learning Analytics
Tore HoelOslo and Akershus University College of Applied Sciences, Oslo, Norway
Lecture & Workshop for PhD students @ ECNU, Shanghai 2014-12-22Course on Smart Education
2
Largest state university college in Norway.
I work mainly with European projectson Learning Analytics and Open Education
About Tore
3
This is more an interactive workshop than a lecture
➔
You have to contribute!
Today’s plan
1. Your own projects on LA and Big Data (paper assignment)
2. Definitions of analytics, academic analytics, learning analytics, etc.
3. Actors in LA
4. Framework models
5. Requirements - the big picture
6. Data and Privacy
4
Learning Analytics and Big Data– Mapping your interest
Related to your selected themes and research goals for your papers on Smart Education
5
What is your concepts of Learning Analytics?
Write down 3 concepts that would be on the top of your list when you will explain what LA is 6
xy
z
Write like aMind Map – in your
own language if you want!
Huaihao Zhang
• Learning analytics: The influence of demographic of K6-9 SL teacher on their engagement in an online teacher training initiative
7
Demographics
Teacher training
Zhenyue Ding
• Subject knowledge bank construction based on Big Data: Framework for describing; Subject Bank; Visualization
• Cloud service platform for K12
• Smart assessment – adaptive assessment for K12
8
Ontology
Visualization
Assessment bank Adaptive assessment
Peter Riezebos
• Understanding LA as educational instrument: methods, ethical issues, optimize learning paths
• Smart assessment: Identify learning outcome, cognitive learning preferences
9
Definition of LA
Ethical guidelines
Learning paths
Learning outcome
Learningpreferences
Huan Liu
• Understanding LA and EDM
• Gathering and coding data
• LA impact on adaptive learning
10
Definition of LA
Definition of EDM
Data
Data metrics
Adaptive
Liang Luo
• Smart pedagogy Instructional Design: Classification of learning activities; learning activity design model
11
Learning ActivityDescription
Learning Design
Concept map example
12Drawn with the Open Source Cmap tool cmap.ihmc.us
Student’s summary of course in LA - work in progress
What is Learning Analytics?
See the LACE FAQ
13
Uploaded to Sakai platform
What are analytics?
• High-level figures
• Brief overview for internal and external reports
• Academic Analytics
• Figures on retention and success, for the institution to assess performance
• Educational Data Mining
• Searching for patterns in the data
• Learning Analytics
• Use of [big] data to provide actionable intelligence for learners and teachers
14
Levels of Learning Analytics
(UNESCO Policy Brief, November 2012)15
Learning Analytics defined
«The measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs.»
Society for Learning Analytics Research (SoLAR)
16
Actionable intelligence!Not
TheoreticalInsights!
Not Reporting!
Clow, LAK12, 2012
When designing a LA system –where should you start?
18
Actors in learning analytics
19
Why do learners use analytics?
• Monitor their own activities and interactions
• Monitor the learning process
• Compare their activity with that of others
• Increase awareness, reflect and self reflect
• Improve discussion participation
• Improve learning behaviour
• Improve performance
• Become better learners
• Learn!20
Why do teachers use analytics?
• Monitor the learning process
• Explore student data
• Identify problems
• Discover patterns
• Find early indicators for success
• Find early indicators for poor marks or drop-out
• Assess usefulness of learning materials
• Increase awareness, reflect and self reflect
• Increase understanding of learning environments
• Intervene, advise and assist
• Improve teaching, resources and the environment
21
Why do learning designers use analytics?
• Helping to identify useful analytics
• What do learners need to know in order to network, collaborate, browse or reflect?
• What do educators need to know to support them?
• Helping to identify gaps in the data
• Which data do we need to collect?
• Helping to identify gaps in our toolkit
• Which design elements can we look at easily?
• Which ones still pose problems?22
More learning design
• Helping to frame and focus analytics questions
• What did they learn?… in relation to learning outcomes
• Were they social?... when they were collaborating
• Did they share links?... when encouraged to browse
• Did they return to steps?... when encouraged to reflect
• Helping to identify appropriate forms of analysis
• The same content, but with a focus on
• Number of visits if content
• Length, quality, number of comments if conversational
• Dwell time and repeat visits if reflection23
Framework Models of LA
24
More models to be found athttp://insulardrafts.tumblr.com
Pick your model and explain to the group what it is all about!
25
26Source: Siemens et al. (2011) Open Learning Anlytics Integrated Platform
28Source: www.laceproject.eu/blog/learning-analytics-research-schools-owd/
29Source: Slides from professor Wu Yonghe, ECNU / ESERC
Requirements for LA
31
The LA feedback loop
32(Greller & Drachsler, 2012)
Critical dimensions of learning analytics (Greller & Drachsler, 2012) 33
34
Data
35
What data are available for LA?
• Data sharing and Privacy Survey
36
The draft questionnaire is uploaded to Sakai. I would like your comments to the questions and ideas how to proceeed!
What data could be used? – some ideas…
• Demographic data
• Calendar information about assignments
• VLE activity data (including forums).
• Lists of required reading
• Library resources usage data
• Library helpdesk enquiries
• Library website usage and analytics data
• Assessment results
• User survey results
• Student retention and attainment data
37Source: Rebecca Ferguson
The concept of Data Commons
38Source: KERIS, Korea
Ideals and Threats
39
What is the ideal use of LA?
• Can we achieve this?
• Aligned with clear aims
• Huge and sustained effort
• Agreed proxies for learning
• Clear and standardised visualisation
• Driving behaviour at every level
• Can we avoid this?
• Instructivist approach
• Stressed, unhappy learners
• Analytics with little value for learners or teachers
• Omission of key areas, such as collaboration
40Source: Rebecca Ferguson, OUUK
Don’t start with the data – start
with the pedagogy
How do people learn?
How can I use data to facilitate that process?
Social learning analytics:
How do people learn socially & in social settings?
How can I use data to facilitate that process?
How could we achieve ideal LA?
41Source: Rebecca Ferguson, OUUK
What questions should I ask?
42
• Which elements are learners struggling with?
• Which sections engage them the most?
• What prompts them to ask questions?
• How are they navigating resources?
• What misconceptions have they shown?
• Are there any accessibility issues?
How can analytics be used to
achieve desired learning outcomes?Source: Rebecca Ferguson, OUUK
How to buildtrust?
43
http://shanghaidaily.com/metro/society/Mini-spies-in-the-classroom-strain-relations/shdaily.shtml
2014-12-22
• Data Protection
• Privacy
• Transparency (related to Subject
Access requests)
• Whether students should be able to
opt in/out
• De-identification of data
• Timeliness and Duty of Care
(keeping data up to date)
• Access to data (who should have
access to the data, etc.)
• Students abusing the system by misinformation
Some ethical challenges
• The use of student data outside
university systems (Social Media)
• Analysis of the data and the methods used (what assumptions are used to create the algorithm for the predictive model, should there be an independent
audit?)
• Purpose of applying a learning analytics
approach
• Profiling of students
• How will it be done?
• What do we tell students?
• Should we tell students? – Students may feel ‘at-risk’/labelled
Glasswinged butterfly, ? Greta oro
cc licensed ( BY NC ND ) flickr photo by Greg Foster: http://www.flickr.com/photos/gregfoster/3365801458/
Thanks to:
• Rebecca Ferguson, OUUK @R3beccaF (for letting me use her slides)
• LACE project colleagues
Funders:
• LACE: European Commission 619424-FP7-ICT-2013-11
Hoel, T. (2014). «Requirements for Learning Analytics» – lecture and workshop at East China Normal University, Shanghai, China, December 2014
Twitter: @tore - WeChat: Tore_noabout.me/[email protected]
This work was undertaken as part of the LACE Project, supported by the European Commission Seventh Framework Programme, grant 619424.
These slides are provided under the Creative Commons Attribution Licence: http://creativecommons.org/licenses/by/4.0/. Some images used may have different licence terms.
www.laceproject.eu@laceproject
46