20180516 learning dashboards tinne de laet...learning analytics design and build analytics...
TRANSCRIPT
![Page 1: 20180516 Learning Dashboards Tinne De Laet...learning analytics design and build analytics dashboards, dashboards that go beyond identifying at-risk students, allowing actionable feedback](https://reader036.vdocuments.mx/reader036/viewer/2022062916/5ec88396f7dbcb479e3953a0/html5/thumbnails/1.jpg)
https://mfeldstein.com/purdue-course-signals-data-issue-explainer/
![Page 2: 20180516 Learning Dashboards Tinne De Laet...learning analytics design and build analytics dashboards, dashboards that go beyond identifying at-risk students, allowing actionable feedback](https://reader036.vdocuments.mx/reader036/viewer/2022062916/5ec88396f7dbcb479e3953a0/html5/thumbnails/2.jpg)
Learning dashboards for first‐year students
the (non)sense of chances of success and predictive models
Tinne De [email protected]
@TinneDeLaet
![Page 3: 20180516 Learning Dashboards Tinne De Laet...learning analytics design and build analytics dashboards, dashboards that go beyond identifying at-risk students, allowing actionable feedback](https://reader036.vdocuments.mx/reader036/viewer/2022062916/5ec88396f7dbcb479e3953a0/html5/thumbnails/3.jpg)
“Learning analytics is about collecting traces that learners leave behind and using those traces to improve learning.”
- Erik Duval
Learning Analytics and Educational Data Mining, Erik Duval’s Weblog, 30 January 2012, https://erikduval.wordpress.com/2012/01/30/learning-analytics-and-educational-data-mining/ 3
Learning Analytics?
![Page 4: 20180516 Learning Dashboards Tinne De Laet...learning analytics design and build analytics dashboards, dashboards that go beyond identifying at-risk students, allowing actionable feedback](https://reader036.vdocuments.mx/reader036/viewer/2022062916/5ec88396f7dbcb479e3953a0/html5/thumbnails/4.jpg)
Learning Dashboards?
4Dashboard Confusion, Stephen Few, Intelligent Enterprise, March 20, 2004
“A dashboard is a visual display of the most important information needed to
achieve one or more objectives; consolidated and arranged on a single
screen so the information can be monitored at a glance.”- Stephen Few
![Page 5: 20180516 Learning Dashboards Tinne De Laet...learning analytics design and build analytics dashboards, dashboards that go beyond identifying at-risk students, allowing actionable feedback](https://reader036.vdocuments.mx/reader036/viewer/2022062916/5ec88396f7dbcb479e3953a0/html5/thumbnails/5.jpg)
Successful Transition from secondary to higher Education using Learning Analytics
enhance a successful transition from secondary to higher education by means of
learning analytics design and build analytics dashboards,
dashboards that go beyond identifying at-risk students, allowing actionable feedback for all
students on a large scale.
Achieving Benefits from Learning Analytics
research strategies and practices for using learning analytics to support students during
their first year at university developing the technological aspects of
learning analytics, focuses on how learning analytics can be used
to support students.
5
www.stela-project.eu
@STELA_project2015‐1‐UK01‐KA203‐013767
www.ableproject.eu
@ABLE_project_eu
562167‐EPP‐1‐2015‐1‐BE‐EPPKA3‐PI‐FORWARD
![Page 6: 20180516 Learning Dashboards Tinne De Laet...learning analytics design and build analytics dashboards, dashboards that go beyond identifying at-risk students, allowing actionable feedback](https://reader036.vdocuments.mx/reader036/viewer/2022062916/5ec88396f7dbcb479e3953a0/html5/thumbnails/6.jpg)
STELA ♥ ABLE
6
actionable feedbackstudent-centered
program level
inclusive
first-year experience
institution-wide
Learning Analytics
actual implementation
![Page 7: 20180516 Learning Dashboards Tinne De Laet...learning analytics design and build analytics dashboards, dashboards that go beyond identifying at-risk students, allowing actionable feedback](https://reader036.vdocuments.mx/reader036/viewer/2022062916/5ec88396f7dbcb479e3953a0/html5/thumbnails/7.jpg)
[!] Feedback must be “actionable”.
7
Warning!
Male students have
10% less probability to
be successful.
You are male.
Warning!
Your online activity is
lagging behind.
action?
?
action?
?
![Page 8: 20180516 Learning Dashboards Tinne De Laet...learning analytics design and build analytics dashboards, dashboards that go beyond identifying at-risk students, allowing actionable feedback](https://reader036.vdocuments.mx/reader036/viewer/2022062916/5ec88396f7dbcb479e3953a0/html5/thumbnails/8.jpg)
8
awareness
(self-)reflection
sensemaking
impact
data
questions
answers
behavior changenew meaning
Verbert K, Duval E, Klerkx J; Govaerts S, Santos JL (2013) Learning analytics dashboard applications. American Behavioural Scientist, 10 pages. Published online February 2013.
[!] Feedback must be “actionable”.
![Page 9: 20180516 Learning Dashboards Tinne De Laet...learning analytics design and build analytics dashboards, dashboards that go beyond identifying at-risk students, allowing actionable feedback](https://reader036.vdocuments.mx/reader036/viewer/2022062916/5ec88396f7dbcb479e3953a0/html5/thumbnails/9.jpg)
learning dashboards @KU Leuven
interaction
self-reflection
LISSA
STUDENTADVISOR STUDENT LASSI –
learning skills
REX - scoresPOS – futurestudents
![Page 10: 20180516 Learning Dashboards Tinne De Laet...learning analytics design and build analytics dashboards, dashboards that go beyond identifying at-risk students, allowing actionable feedback](https://reader036.vdocuments.mx/reader036/viewer/2022062916/5ec88396f7dbcb479e3953a0/html5/thumbnails/10.jpg)
[!] Start with the available data.
Lots of data may eventually become available in the future …
…. already start with what is available
10
(*)
(*) Zarraonandia, T., Aedo, I., Díaz, P., & Montero, A. (2013). An augmented lecture feedback system to support learner and teacher communication.British Journal of Educational Technology, 44(4), 616-628.
![Page 11: 20180516 Learning Dashboards Tinne De Laet...learning analytics design and build analytics dashboards, dashboards that go beyond identifying at-risk students, allowing actionable feedback](https://reader036.vdocuments.mx/reader036/viewer/2022062916/5ec88396f7dbcb479e3953a0/html5/thumbnails/11.jpg)
Case studydashboard interaction student – study advisor
![Page 12: 20180516 Learning Dashboards Tinne De Laet...learning analytics design and build analytics dashboards, dashboards that go beyond identifying at-risk students, allowing actionable feedback](https://reader036.vdocuments.mx/reader036/viewer/2022062916/5ec88396f7dbcb479e3953a0/html5/thumbnails/12.jpg)
Study advisor – student conversations
12
Should I consideranother program?
Can I still finish the bachelor in 3 years?
How should I composemy program for next year?
What is the personalsituation?
How can I help?
What is the best next step?
![Page 13: 20180516 Learning Dashboards Tinne De Laet...learning analytics design and build analytics dashboards, dashboards that go beyond identifying at-risk students, allowing actionable feedback](https://reader036.vdocuments.mx/reader036/viewer/2022062916/5ec88396f7dbcb479e3953a0/html5/thumbnails/13.jpg)
[!] Use all available expertise.
13
visualization experts
practitioners / end-users
researchers LA
researchers first-year study success
Charleer S., Vande Moere A., Klerkx J., Verbert K., De Laet T. (2017). Learning Analytics Dashboards to Support Adviser-Student Dialogue.In IEEE Transactions on Learning Technology (http://ieeexplore.ieee.org/document/7959628/).
![Page 14: 20180516 Learning Dashboards Tinne De Laet...learning analytics design and build analytics dashboards, dashboards that go beyond identifying at-risk students, allowing actionable feedback](https://reader036.vdocuments.mx/reader036/viewer/2022062916/5ec88396f7dbcb479e3953a0/html5/thumbnails/14.jpg)
LISSA dashboardhttps://able.cs.kuleuven.be/demo‐september/2016/2
![Page 15: 20180516 Learning Dashboards Tinne De Laet...learning analytics design and build analytics dashboards, dashboards that go beyond identifying at-risk students, allowing actionable feedback](https://reader036.vdocuments.mx/reader036/viewer/2022062916/5ec88396f7dbcb479e3953a0/html5/thumbnails/15.jpg)
[!] Wording matters.
15
73% chance of success
73% of students of earlier cohorts with the same
study efficiency obtained the bachelor degree
http://blog.associatie.kuleuven.be/tinnedelaet/the-nonsense-of-chances-of-success-and-predictive-models/
![Page 16: 20180516 Learning Dashboards Tinne De Laet...learning analytics design and build analytics dashboards, dashboards that go beyond identifying at-risk students, allowing actionable feedback](https://reader036.vdocuments.mx/reader036/viewer/2022062916/5ec88396f7dbcb479e3953a0/html5/thumbnails/16.jpg)
LISSA dashboard
16
Three examination periods
observations, interviews, questionnaires
pilot with two engineering programs
Charleer S., Vande Moere A., Klerkx J., Verbert K., De Laet T. (2017). Learning Analytics Dashboards to Support Adviser-Student Dialogue. In IEEE Transactions on Learning Technology
![Page 17: 20180516 Learning Dashboards Tinne De Laet...learning analytics design and build analytics dashboards, dashboards that go beyond identifying at-risk students, allowing actionable feedback](https://reader036.vdocuments.mx/reader036/viewer/2022062916/5ec88396f7dbcb479e3953a0/html5/thumbnails/17.jpg)
LISSA: evaluation – observations
17
15 observations
insights(-) factual(+) interpretative (!) reflective
Charleer S., Vande Moere A., Klerkx J., Verbert K., De Laet T. (2017). Learning Analytics Dashboards to Support Adviser-Student Dialogue. In IEEE Transactions on Learning Technology
![Page 18: 20180516 Learning Dashboards Tinne De Laet...learning analytics design and build analytics dashboards, dashboards that go beyond identifying at-risk students, allowing actionable feedback](https://reader036.vdocuments.mx/reader036/viewer/2022062916/5ec88396f7dbcb479e3953a0/html5/thumbnails/18.jpg)
Evaluation – interviews
“When students see the numbers, they are surprised, but now they believe me.
Before, I used my gut feeling, now I feelmore certain of what I say as well”.
“It’s like a main thread guiding the
conversation.”
“I can talk about what to do with the results, instead of each time looking for the data and
puzzling it together.”
“Students don’t know where to look during the conversation, and avoid eye contact.
The dashboard provides them a point of focus”.
“A student changed her study method in June and could now see it paid off.”
LISSA supports a personal dialogue.
the level of usage depends on the experience and style of the study advisors
fact-based evidence at the side narrative thread
key moments and student path help to reconstruct personal track
“I can focus on the student’s personal
path, rather than on the facts.”
“Now, I can blamethe dashboard and
focus on collaboratively looking
for the next step to take.”
18
![Page 19: 20180516 Learning Dashboards Tinne De Laet...learning analytics design and build analytics dashboards, dashboards that go beyond identifying at-risk students, allowing actionable feedback](https://reader036.vdocuments.mx/reader036/viewer/2022062916/5ec88396f7dbcb479e3953a0/html5/thumbnails/19.jpg)
LISSA: status
19
26 programs >4500 students114 student advisors
training of study advisors
http://blog.associatie.kuleuven.be/tinnedelaet/lissa-learning-dashboard-supporting-student-advisers-in-traditional-higher-education/
Millecamp M., Gutiérrez F., Charleer S., Verbert K., De Laet T.# (2018). A qualitative evaluation of a learning dashboard to support advisor-student dialogues. Proceedings of the 8th International Learning Analytics & Knowledge Conference. LAK. Sydney, 5-9 March 2018 (pp. 1-5) ACM.
dashboards for three examination periods
![Page 20: 20180516 Learning Dashboards Tinne De Laet...learning analytics design and build analytics dashboards, dashboards that go beyond identifying at-risk students, allowing actionable feedback](https://reader036.vdocuments.mx/reader036/viewer/2022062916/5ec88396f7dbcb479e3953a0/html5/thumbnails/20.jpg)
LISSA: evaluation – student questionnaires
20
26 programs @KU Leuven291 student questionnaires
first examination period
“Confronting, but useful”
“I want to use thisdashboard at home.”
“Also show the sub‐gradesfor labs, … ”
“How can I know the data is trustworth?”
“Can’t these visualizations besend to students?” “Crisp and clear.”
![Page 21: 20180516 Learning Dashboards Tinne De Laet...learning analytics design and build analytics dashboards, dashboards that go beyond identifying at-risk students, allowing actionable feedback](https://reader036.vdocuments.mx/reader036/viewer/2022062916/5ec88396f7dbcb479e3953a0/html5/thumbnails/21.jpg)
21
0
0
1
1
1
1
4
2
1
4
4
3
29
21
36
37
49
42
176
112
156
132
141
169
80
155
93
116
92
72
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
1. The dashboard is clarifying and surveyable.
2. The shown information regarding my studysituation is correct.
3. The shown position with respect to my fellowstudents (histograms per exam and global…
4. A conversation with my student advisors helpedme to gain insight in my study trajectory.
5. The visualisation is of added value to theconversation with the student advisor.
6. The shown information provide me insight inmy current situation.
Student questionnaire January 2018 (N=291)
Strongly Disagree Disagree Neither Agree or Disagree Agree Strongly Agree
![Page 22: 20180516 Learning Dashboards Tinne De Laet...learning analytics design and build analytics dashboards, dashboards that go beyond identifying at-risk students, allowing actionable feedback](https://reader036.vdocuments.mx/reader036/viewer/2022062916/5ec88396f7dbcb479e3953a0/html5/thumbnails/22.jpg)
[!] Do not oversimplify. Show uncertainty.
22
• reality is complex
• measurement is limited
• individual circumstances
• need for nuance
• trigger reflection
http://blog.associatie.kuleuven.be/tinnedelaet/the-nonsense-of-chances-of-success-and-predictive-models/
![Page 23: 20180516 Learning Dashboards Tinne De Laet...learning analytics design and build analytics dashboards, dashboards that go beyond identifying at-risk students, allowing actionable feedback](https://reader036.vdocuments.mx/reader036/viewer/2022062916/5ec88396f7dbcb479e3953a0/html5/thumbnails/23.jpg)
[!] Be careful with predictive algorithms.
23http://blog.associatie.kuleuven.be/tinnedelaet/the-nonsense-of-chances-of-success-and-predictive-models/
• reality is complex
• measurement is limited
• individual circumstances
• need for nuance
• trigger reflection
![Page 24: 20180516 Learning Dashboards Tinne De Laet...learning analytics design and build analytics dashboards, dashboards that go beyond identifying at-risk students, allowing actionable feedback](https://reader036.vdocuments.mx/reader036/viewer/2022062916/5ec88396f7dbcb479e3953a0/html5/thumbnails/24.jpg)
Case studystudent-facing dashboards
![Page 25: 20180516 Learning Dashboards Tinne De Laet...learning analytics design and build analytics dashboards, dashboards that go beyond identifying at-risk students, allowing actionable feedback](https://reader036.vdocuments.mx/reader036/viewer/2022062916/5ec88396f7dbcb479e3953a0/html5/thumbnails/25.jpg)
[!] Start with the available data.
25
data already available?
administrative (examples)
student records course grades
systems (examples)
LMS access logs advisor meetings
)
Broos T., Verbert K., Van Soom C., Langie G., De Laet T.# (2018). Small data as a conversation starter for learning analytics: exam results dashboard for first-year students in higher education. Journal of Research in Innovative Teaching & Learning, , 1-14.
![Page 26: 20180516 Learning Dashboards Tinne De Laet...learning analytics design and build analytics dashboards, dashboards that go beyond identifying at-risk students, allowing actionable feedback](https://reader036.vdocuments.mx/reader036/viewer/2022062916/5ec88396f7dbcb479e3953a0/html5/thumbnails/26.jpg)
[!] Think beyond the obvious data.
26
• Don’t think too traditional.
• Many institutions are collecting survey data for educational research.
![Page 27: 20180516 Learning Dashboards Tinne De Laet...learning analytics design and build analytics dashboards, dashboards that go beyond identifying at-risk students, allowing actionable feedback](https://reader036.vdocuments.mx/reader036/viewer/2022062916/5ec88396f7dbcb479e3953a0/html5/thumbnails/27.jpg)
[!] Not all data is usable.
27
example data from a traditional course with “VLE as a file system”
test scores
activity/week (#days)
weeks of the year
![Page 28: 20180516 Learning Dashboards Tinne De Laet...learning analytics design and build analytics dashboards, dashboards that go beyond identifying at-risk students, allowing actionable feedback](https://reader036.vdocuments.mx/reader036/viewer/2022062916/5ec88396f7dbcb479e3953a0/html5/thumbnails/28.jpg)
[!] Not all data is usable.
28
example data from a course with flipped classroom & blended learning
exam scores
activity (# of modules used)
Not a single student using less than 10
modules passed the course.
Most of the successful students used 15 modules or more.
![Page 29: 20180516 Learning Dashboards Tinne De Laet...learning analytics design and build analytics dashboards, dashboards that go beyond identifying at-risk students, allowing actionable feedback](https://reader036.vdocuments.mx/reader036/viewer/2022062916/5ec88396f7dbcb479e3953a0/html5/thumbnails/29.jpg)
[!] Keep Learning Analytics in mind when designing learning activities.
29
Learning Analytics Learning Design
INFORM
ENABLE
If LA indeed contributes to improved learning design…
… don’t make it an afterthought
![Page 30: 20180516 Learning Dashboards Tinne De Laet...learning analytics design and build analytics dashboards, dashboards that go beyond identifying at-risk students, allowing actionable feedback](https://reader036.vdocuments.mx/reader036/viewer/2022062916/5ec88396f7dbcb479e3953a0/html5/thumbnails/30.jpg)
30
Does my concentrationmatter?
How is my time management?
I feel uncertain.Is this normal?
How can I improve my concentration?
![Page 31: 20180516 Learning Dashboards Tinne De Laet...learning analytics design and build analytics dashboards, dashboards that go beyond identifying at-risk students, allowing actionable feedback](https://reader036.vdocuments.mx/reader036/viewer/2022062916/5ec88396f7dbcb479e3953a0/html5/thumbnails/31.jpg)
data already available?
administrative (examples)
student records course grades
[!] Think beyond the obvious data.
31
systems (examples)
LMS access logs advisor meetings
surveys (examples)
quality insurance LASSI
![Page 32: 20180516 Learning Dashboards Tinne De Laet...learning analytics design and build analytics dashboards, dashboards that go beyond identifying at-risk students, allowing actionable feedback](https://reader036.vdocuments.mx/reader036/viewer/2022062916/5ec88396f7dbcb479e3953a0/html5/thumbnails/32.jpg)
~ 30 LASSI questions(shortened version)
“Learning Skills”
Example: When preparing for an exam, I create questions that I think might be included.
Example: I find it difficult to maintain my concentration while doing my coursework.
Example: I find it hard to stick to a study schedule.
raw scores(selected 5 out of 10)
CONCENTRATION
MOTIVATION
FAILURE ANXIETY
TEST STRATEGY
TIME MANAGEMENT
norm scores(in Flemish HE context)
Example: STRONG
Example: AVERAGE
Example: LOW
Example: VERY STRONG
Example: VERY WEAK
32
Meta cognitive abilities
Pinxten, M., Van Soom, C., Peeters, C., De Laet, T., Langie, G., At-risk at the gate: prediction of study success of first-year science and engineering students in an open-admission university in Flanders—any incremental validity of study strategies? Eur J Psychol Educ (2017). readySTEMgo Erasmus+ project https://iiw.kuleuven.be/english/readystemgo
![Page 33: 20180516 Learning Dashboards Tinne De Laet...learning analytics design and build analytics dashboards, dashboards that go beyond identifying at-risk students, allowing actionable feedback](https://reader036.vdocuments.mx/reader036/viewer/2022062916/5ec88396f7dbcb479e3953a0/html5/thumbnails/33.jpg)
Dashboard learning skills
33
students complete LASSI questionnaire
students received personalized emailwith invitation for dashboard
4367 students in 26 programsin 9 faculties @KU Leuven
demo: https://learninganalytics.set.kuleuven.be/lassi-1718/ (KU Leuven login)
2 programs @TU Delft
![Page 34: 20180516 Learning Dashboards Tinne De Laet...learning analytics design and build analytics dashboards, dashboards that go beyond identifying at-risk students, allowing actionable feedback](https://reader036.vdocuments.mx/reader036/viewer/2022062916/5ec88396f7dbcb479e3953a0/html5/thumbnails/34.jpg)
Feedback model
1. What is this about?
2. How am I doing?
3. How does this relates to others?
4. Why is this relevant?
5. What can I do about it?
34
![Page 35: 20180516 Learning Dashboards Tinne De Laet...learning analytics design and build analytics dashboards, dashboards that go beyond identifying at-risk students, allowing actionable feedback](https://reader036.vdocuments.mx/reader036/viewer/2022062916/5ec88396f7dbcb479e3953a0/html5/thumbnails/35.jpg)
35
3. How does this relates to others?
2. How am I doing?
1. What is this about?
@studyProgram@
@yourScore@
![Page 36: 20180516 Learning Dashboards Tinne De Laet...learning analytics design and build analytics dashboards, dashboards that go beyond identifying at-risk students, allowing actionable feedback](https://reader036.vdocuments.mx/reader036/viewer/2022062916/5ec88396f7dbcb479e3953a0/html5/thumbnails/36.jpg)
4. Why is this relevant?
5. What can I do about it?
36
![Page 37: 20180516 Learning Dashboards Tinne De Laet...learning analytics design and build analytics dashboards, dashboards that go beyond identifying at-risk students, allowing actionable feedback](https://reader036.vdocuments.mx/reader036/viewer/2022062916/5ec88396f7dbcb479e3953a0/html5/thumbnails/37.jpg)
37
5. What can I do about it?
![Page 38: 20180516 Learning Dashboards Tinne De Laet...learning analytics design and build analytics dashboards, dashboards that go beyond identifying at-risk students, allowing actionable feedback](https://reader036.vdocuments.mx/reader036/viewer/2022062916/5ec88396f7dbcb479e3953a0/html5/thumbnails/38.jpg)
Response
38
3868 students (89%) used dashboard
![Page 39: 20180516 Learning Dashboards Tinne De Laet...learning analytics design and build analytics dashboards, dashboards that go beyond identifying at-risk students, allowing actionable feedback](https://reader036.vdocuments.mx/reader036/viewer/2022062916/5ec88396f7dbcb479e3953a0/html5/thumbnails/39.jpg)
Student feedback?
39http://blog.associatie.kuleuven.be/tinnedelaet/learning-dashboard-for-actionable-feedback-on-learning-and-studying-skills/
How CLEAR is this info?
stars stars
![Page 40: 20180516 Learning Dashboards Tinne De Laet...learning analytics design and build analytics dashboards, dashboards that go beyond identifying at-risk students, allowing actionable feedback](https://reader036.vdocuments.mx/reader036/viewer/2022062916/5ec88396f7dbcb479e3953a0/html5/thumbnails/40.jpg)
Students that click through
Broos, T., Peeters, L., Verbert, K., Van Soom, C., Langie, G., & De Laet, T. (2017, July). Dashboard for Actionable Feedback on Learning Skills: Scalability and Usefulness.In International Conference on Learning and Collaboration Technologies (pp. 229-241). Springer, Cham.
40
better learning skills
![Page 41: 20180516 Learning Dashboards Tinne De Laet...learning analytics design and build analytics dashboards, dashboards that go beyond identifying at-risk students, allowing actionable feedback](https://reader036.vdocuments.mx/reader036/viewer/2022062916/5ec88396f7dbcb479e3953a0/html5/thumbnails/41.jpg)
More intense users
Broos, T., Peeters, L., Verbert, K., Van Soom, C., Langie, G., & De Laet, T. (2017, July). Dashboard for Actionable Feedback on Learning Skills: Scalability and Usefulness.In International Conference on Learning and Collaboration Technologies (pp. 229-241). Springer, Cham.
41
worse learning skills
![Page 42: 20180516 Learning Dashboards Tinne De Laet...learning analytics design and build analytics dashboards, dashboards that go beyond identifying at-risk students, allowing actionable feedback](https://reader036.vdocuments.mx/reader036/viewer/2022062916/5ec88396f7dbcb479e3953a0/html5/thumbnails/42.jpg)
[!] Give students “the key”.
42
• Student has the key to own data.
• Student takes initiative to share/discuss own data.
• GDPR as opportunity!
![Page 43: 20180516 Learning Dashboards Tinne De Laet...learning analytics design and build analytics dashboards, dashboards that go beyond identifying at-risk students, allowing actionable feedback](https://reader036.vdocuments.mx/reader036/viewer/2022062916/5ec88396f7dbcb479e3953a0/html5/thumbnails/43.jpg)
Dashboard positioning test
43https://feedback.ijkingstoets.be/ijkingstoets-10-ir/index.html (10ir0demo)
![Page 44: 20180516 Learning Dashboards Tinne De Laet...learning analytics design and build analytics dashboards, dashboards that go beyond identifying at-risk students, allowing actionable feedback](https://reader036.vdocuments.mx/reader036/viewer/2022062916/5ec88396f7dbcb479e3953a0/html5/thumbnails/44.jpg)
[!] Acceptance precedes impact.
44
• Involve stakeholders from the start and value their input!
COmmunicationCOoperation
• Demonstrate usefulness.
• Take care of ethics and privacy.
• Best scenario: students & study advisors as ambassadors
CO
CO
![Page 45: 20180516 Learning Dashboards Tinne De Laet...learning analytics design and build analytics dashboards, dashboards that go beyond identifying at-risk students, allowing actionable feedback](https://reader036.vdocuments.mx/reader036/viewer/2022062916/5ec88396f7dbcb479e3953a0/html5/thumbnails/45.jpg)
Impact?
survey before intervention 2nd year students 2016-2017 experiences first-year feedback 41 vragen, 5-point Likert scale pen & paper
dashboards LISSA LASSI (learning skills) 3 x REX (grades)
Survey after intervention 2nd year students 2017-2018
![Page 46: 20180516 Learning Dashboards Tinne De Laet...learning analytics design and build analytics dashboards, dashboards that go beyond identifying at-risk students, allowing actionable feedback](https://reader036.vdocuments.mx/reader036/viewer/2022062916/5ec88396f7dbcb479e3953a0/html5/thumbnails/46.jpg)
Impact?
During the first year I received sufficient information regarding my academic achievements.
46
Engineering Science (p<0.001)
![Page 47: 20180516 Learning Dashboards Tinne De Laet...learning analytics design and build analytics dashboards, dashboards that go beyond identifying at-risk students, allowing actionable feedback](https://reader036.vdocuments.mx/reader036/viewer/2022062916/5ec88396f7dbcb479e3953a0/html5/thumbnails/47.jpg)
Impact?
The information I received helped to position myself with respect to my peers.
47
Engineering Science (p<0.001)
![Page 48: 20180516 Learning Dashboards Tinne De Laet...learning analytics design and build analytics dashboards, dashboards that go beyond identifying at-risk students, allowing actionable feedback](https://reader036.vdocuments.mx/reader036/viewer/2022062916/5ec88396f7dbcb479e3953a0/html5/thumbnails/48.jpg)
Impact?
48
The information I received made me reflect.
The information I received made me adapt my behaviour.
![Page 49: 20180516 Learning Dashboards Tinne De Laet...learning analytics design and build analytics dashboards, dashboards that go beyond identifying at-risk students, allowing actionable feedback](https://reader036.vdocuments.mx/reader036/viewer/2022062916/5ec88396f7dbcb479e3953a0/html5/thumbnails/49.jpg)
[!] Context matters!
• available data
• national and institutional regulations and culture
• educational vision
• educational system, size of population ..
• …
Don’t just copy existing LA solutions!
49
![Page 50: 20180516 Learning Dashboards Tinne De Laet...learning analytics design and build analytics dashboards, dashboards that go beyond identifying at-risk students, allowing actionable feedback](https://reader036.vdocuments.mx/reader036/viewer/2022062916/5ec88396f7dbcb479e3953a0/html5/thumbnails/50.jpg)
Summary
case studies 11 findings/recommendations
[!] Use all available expertise.
[!] Start with the available data.
[!] Look beyond the obvious data.
[!] Not all data is usable.
[!] Wording matters.
[!] Don’t oversimplify. Show uncertainty.
[!] Beware of predictive algorithms.
[!] Keep Learning Analytics in mind when designing
learning activities.
[!] Give students “the key” to their data.
[!] Acceptance precedes impact.
[!] Context matters!
humble approach
small data
involvement of stakeholders, especially practitioners
actionable feedback
scalability
traditional university settings
Is this Learning Analytics?
![Page 51: 20180516 Learning Dashboards Tinne De Laet...learning analytics design and build analytics dashboards, dashboards that go beyond identifying at-risk students, allowing actionable feedback](https://reader036.vdocuments.mx/reader036/viewer/2022062916/5ec88396f7dbcb479e3953a0/html5/thumbnails/51.jpg)
Future?
51
Continue and extend dashboards @KU Leuven?
Transfer to other universities?
extension?
![Page 52: 20180516 Learning Dashboards Tinne De Laet...learning analytics design and build analytics dashboards, dashboards that go beyond identifying at-risk students, allowing actionable feedback](https://reader036.vdocuments.mx/reader036/viewer/2022062916/5ec88396f7dbcb479e3953a0/html5/thumbnails/52.jpg)
Project team @
52
Sven CharleerAugmentHCI, Computer Science department
PhD researcher ABLE
Katrien VerbertAugmentHCI, Computer Science department
Copromotor of STELA & ABLE
Carolien Van SoomLeuven Engineering and Science Education CenterHead of Tutorial Services of ScienceCopromotor of STELA & ABLE
Greet Langie Leuven Engineering and Science Education Center
Vicedean (education) faculty of Engineering TechnologyCopromotor of STELA & ABLE
Tinne De LaetLeuven Engineering and Science Education CenterHead of Tutorial Services of Engineering ScienceCoordinator of STELAKU Leuven coordinator of ABLE
Francisco GutiérrezAugmentHCI, Computer Science departmentPhD researcher ABLE
Tom BroosLeuven Engineering and Science Education CenterAugmentHCI, Computer Science departmentPhD researcher STELA
Martijn MillecampAugmentHCI, Computer Science department
PhD researcher ABLE
Special thanks to study advisors for their cooperation, advice, feedback, and support!Jasper, Bart, Riet, Hilde, An, Katrien, …
♥
![Page 53: 20180516 Learning Dashboards Tinne De Laet...learning analytics design and build analytics dashboards, dashboards that go beyond identifying at-risk students, allowing actionable feedback](https://reader036.vdocuments.mx/reader036/viewer/2022062916/5ec88396f7dbcb479e3953a0/html5/thumbnails/53.jpg)
53
2
3
1
7
2
1
10
44
23
36
3
11
64
97
81
74
36
29
150
115
126
110
119
91
61
30
56
59
128
157
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
7. The dashboard makes me more aware on my currentstudy situation.
8. The dashboard makes me forecast the differentpossibilities in my future study trajectory.
9. The dashboard helps me to reflect on my past andcurrent study behaviour or study trajectory.
10. The dashboards stimulates me to adapt my approachin my studies for the future (study behaviour or study…
11. If I will have a new conversation after one of the nextexamination periods, I hope that the visualisation will be…
12. I would like to consult the information on my own.
Student questionnaire January 2018 (N=291)
Strongly Disagree Disagree Neither Agree or Disagree Agree Strongly Agree
![Page 54: 20180516 Learning Dashboards Tinne De Laet...learning analytics design and build analytics dashboards, dashboards that go beyond identifying at-risk students, allowing actionable feedback](https://reader036.vdocuments.mx/reader036/viewer/2022062916/5ec88396f7dbcb479e3953a0/html5/thumbnails/54.jpg)
How to determine thresholds for different
groups?
LISSA dashboard
54
upper and lower group: clear message
middle group as small as possible
Do not overfit! (nuance)