the role of data in the provision of feedback at scale
TRANSCRIPT
![Page 1: The role of data in the provision of feedback at scale](https://reader036.vdocuments.mx/reader036/viewer/2022062503/58f9b7cc1a28ab0c238b45cf/html5/thumbnails/1.jpg)
Abelardo Pardo (@abelardopardo)Faculty of Engineering and IT
slideshare.net/abelardo_pardo
The role of data in the provision of feedback at scale
Wally G
obetz flickr.com
ASCILITE Webinar 11 April 2017
![Page 2: The role of data in the provision of feedback at scale](https://reader036.vdocuments.mx/reader036/viewer/2022062503/58f9b7cc1a28ab0c238b45cf/html5/thumbnails/2.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 2
Learning Analytics
Situated Actions
Feedback Examples
theilr flickr.com
![Page 3: The role of data in the provision of feedback at scale](https://reader036.vdocuments.mx/reader036/viewer/2022062503/58f9b7cc1a28ab0c238b45cf/html5/thumbnails/3.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 3
LearningAnalytics
SituatedActions
theilr flickr.com
![Page 4: The role of data in the provision of feedback at scale](https://reader036.vdocuments.mx/reader036/viewer/2022062503/58f9b7cc1a28ab0c238b45cf/html5/thumbnails/4.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 4
Dm
itry Grigoriev flickr.com
Learning Analytics: measure, collect, analyse data about learners to understand and improve their learning and the environment in which it occurs
![Page 5: The role of data in the provision of feedback at scale](https://reader036.vdocuments.mx/reader036/viewer/2022062503/58f9b7cc1a28ab0c238b45cf/html5/thumbnails/5.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 5
Derek Bruff flickr.comUnderstanding and
Improving Learning
Algorithms
Dashboards
Clickstreams
Multimodal
SNA
Writing analytics
Learning Designs
![Page 6: The role of data in the provision of feedback at scale](https://reader036.vdocuments.mx/reader036/viewer/2022062503/58f9b7cc1a28ab0c238b45cf/html5/thumbnails/6.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 6
James Rhodes flickr.comLet’s not forget:
Learning analytics are about learning
Gašević, D., Dawson, S., & Siemens, G. (2015). Let's not forget: Learning analytics are about learning. TechTrends, 59(1), 64-75.
![Page 7: The role of data in the provision of feedback at scale](https://reader036.vdocuments.mx/reader036/viewer/2022062503/58f9b7cc1a28ab0c238b45cf/html5/thumbnails/7.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 7
Derek Bruff flickr.com
Learning is situated in interactions with others and relationships to the environment
Ellis, R. A., & Goodyear, P. (2013). Students' Experiences of E-learning in Higher Education. New York and London: Routledge
![Page 8: The role of data in the provision of feedback at scale](https://reader036.vdocuments.mx/reader036/viewer/2022062503/58f9b7cc1a28ab0c238b45cf/html5/thumbnails/8.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 8
Chris M
oore flickr.com
Learning environment has a profound effect
on how data is analysed
![Page 9: The role of data in the provision of feedback at scale](https://reader036.vdocuments.mx/reader036/viewer/2022062503/58f9b7cc1a28ab0c238b45cf/html5/thumbnails/9.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 9
senscience flickr.com
Marzouk, Z., Rokovic, M., Liaqat, A., Vytasek, J., Samadi, D., Stewart-Alonso, J., . . . Nesbit, J. C. (2016). What if learning analytics were based on learning science? Australasian Journal of Educational Technology, 32(6).
• Setting goals & monitoring progress • Distributed practice • Retrieval practice • Prior knowledge for reading • Comparative evaluation of writing • Collaborative Learning
![Page 10: The role of data in the provision of feedback at scale](https://reader036.vdocuments.mx/reader036/viewer/2022062503/58f9b7cc1a28ab0c238b45cf/html5/thumbnails/10.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 10
Joshua Rappeneker flickr.com
Data Understanding
Actions
Improving
![Page 11: The role of data in the provision of feedback at scale](https://reader036.vdocuments.mx/reader036/viewer/2022062503/58f9b7cc1a28ab0c238b45cf/html5/thumbnails/11.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 11
Learning Analytics Model
![Page 12: The role of data in the provision of feedback at scale](https://reader036.vdocuments.mx/reader036/viewer/2022062503/58f9b7cc1a28ab0c238b45cf/html5/thumbnails/12.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 12
Learning Analytics
Situated Actions
Feedback
theilr flickr.com
![Page 13: The role of data in the provision of feedback at scale](https://reader036.vdocuments.mx/reader036/viewer/2022062503/58f9b7cc1a28ab0c238b45cf/html5/thumbnails/13.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 13Hattie, J. A. (1999). Influences on student learning. Inaugural professorial address, University of Auckland, New Zealand
If you could choose one…• More than 500 meta-analyses
of student achievements
• 100 factors with potential influence
• Feedback in top five
• (74 meta-analyses) Most effective form: video, audio, computer-assisted instructional feedback, and/or related goals
![Page 14: The role of data in the provision of feedback at scale](https://reader036.vdocuments.mx/reader036/viewer/2022062503/58f9b7cc1a28ab0c238b45cf/html5/thumbnails/14.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 14
Paula Grubb flickr.com
"Feedback is a process whereby learners obtain information about their work in order to appreciate the similarities and differences between the appropriate standards for any given work, and the qualities of the work itself, in order to generate improved work"
Boud, D., & Molloy, E. (Eds.). (2013). Feedback in Higher and Professional Education: Understanding it and doing it well. London and New York: Routledge.
![Page 15: The role of data in the provision of feedback at scale](https://reader036.vdocuments.mx/reader036/viewer/2022062503/58f9b7cc1a28ab0c238b45cf/html5/thumbnails/15.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 15
Hattie, J., & Timperley, H. (2007). The Power of Feedback. Review of Educational Research, 77(1), 81-112. doi:10.3102/003465430298487
Chris Ballard flickr.com
Feedback Levels
1. Task Level (understanding, performance)
2. Process Level (what to do to understand, perform)
3. Self-regulation level (detecting and directing effort)
4. Self level (personal evaluation and affect)
![Page 16: The role of data in the provision of feedback at scale](https://reader036.vdocuments.mx/reader036/viewer/2022062503/58f9b7cc1a28ab0c238b45cf/html5/thumbnails/16.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 16
Nicol, D., & Macfarlane‐Dick, D. (2006). Formative assessment and self‐regulated learning: a model and seven principles of good feedback practice. Studies in Higher Education, 31(2), 199-218. doi:10.1080/03075070600572090
![Page 17: The role of data in the provision of feedback at scale](https://reader036.vdocuments.mx/reader036/viewer/2022062503/58f9b7cc1a28ab0c238b45cf/html5/thumbnails/17.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale
Good feedback practice1. Clarify what good performance is
2. Facilitate self-assessment
3. Deliver high quality feedback information
4. Encourage teacher and peer dialogue
5. Encourage positive motivation and self-esteem
6. Provide opportunities to close the gap
7. Use information to improve teaching
17
Nicol, D., & Macfarlane‐Dick, D. (2006). Formative assessment and self‐regulated learning: a model and seven principles of good feedback practice. Studies in Higher Education, 31(2), 199-218. doi:10.1080/03075070600572090
![Page 18: The role of data in the provision of feedback at scale](https://reader036.vdocuments.mx/reader036/viewer/2022062503/58f9b7cc1a28ab0c238b45cf/html5/thumbnails/18.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 18
Baker, R. S. (2016). Stupid Tutoring Systems, Intelligent Humans. International Journal of Artificial Intelligence in Education. doi:10.1007/s40593-016-0105-0
Rienties, B., Boroowa, A., Cross, S., Kubiak, C., Mayles, K., & Murphy, S. (2016). Analytics4Action Evaluation Framework: A Review of Evidence-Based Learning Analytics Interventions at the Open University UK. Journal of Interactive Media in Education, 2016(1). doi:10.5334/jime.394
Change in focus, from artificial intelligence to intelligence amplification
Tutors are informed about students struggling but it is up to them to decide if/when to act
![Page 19: The role of data in the provision of feedback at scale](https://reader036.vdocuments.mx/reader036/viewer/2022062503/58f9b7cc1a28ab0c238b45cf/html5/thumbnails/19.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 19
Learning Analytics
Situated Actions
Feedback Examples
theilr flickr.com
![Page 20: The role of data in the provision of feedback at scale](https://reader036.vdocuments.mx/reader036/viewer/2022062503/58f9b7cc1a28ab0c238b45cf/html5/thumbnails/20.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 20
Tracking
![Page 21: The role of data in the provision of feedback at scale](https://reader036.vdocuments.mx/reader036/viewer/2022062503/58f9b7cc1a28ab0c238b45cf/html5/thumbnails/21.jpg)
Abelardo Pardo Can data help me learn? 21
Jovanovic, J., Gašević, D., Pardo, A., Dawson, S., & Mirriahi, N. (In Press. Accepted 20/Feb/2017). Learning Analytics to Unveil Learning Strategies in a Flipped Classroom. Internet and Higher Education. doi:10.1016/j.iheduc.2017.02.001
Analyse Learning Strategies
![Page 22: The role of data in the provision of feedback at scale](https://reader036.vdocuments.mx/reader036/viewer/2022062503/58f9b7cc1a28ab0c238b45cf/html5/thumbnails/22.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 22
Report to students
Kahn, I., & Pardo, A. (2016). Data2U: Scalable Real time Student Feedback in Active Learning Environments. Paper presented at the International Conference on Learning Analytics and Knowledge, Edinburgh, UK. doi:10.1145/2883851.2883911
![Page 23: The role of data in the provision of feedback at scale](https://reader036.vdocuments.mx/reader036/viewer/2022062503/58f9b7cc1a28ab0c238b45cf/html5/thumbnails/23.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 23
No statistically significant difference in the rating of feedback (2013 edition, M=3.25, SD=0.97; 2014 edition, M=3.35, SD=1.03); t(389.78) = -0.97, p <0.17
![Page 24: The role of data in the provision of feedback at scale](https://reader036.vdocuments.mx/reader036/viewer/2022062503/58f9b7cc1a28ab0c238b45cf/html5/thumbnails/24.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 24
You should take a more careful look at how symbols are encoded in the video. Would you be able to encode/decode UAL symbols without looking at the video?
Good initial work. However, did you understand the trick to handle encoding with a variable number of bits? Would you be able to provide an example?
Good work. Would you be able to come up with your own machine language and your encoding scheme? Remember that it has to be unambiguous.
Thorough work with the task about machine language encoding. Give it a quick review before the midterm.
Q1 Q2 Q3 Q4
Instructor
![Page 25: The role of data in the provision of feedback at scale](https://reader036.vdocuments.mx/reader036/viewer/2022062503/58f9b7cc1a28ab0c238b45cf/html5/thumbnails/25.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 25
Algorithm
![Page 26: The role of data in the provision of feedback at scale](https://reader036.vdocuments.mx/reader036/viewer/2022062503/58f9b7cc1a28ab0c238b45cf/html5/thumbnails/26.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 26
AutomaticEmail
![Page 27: The role of data in the provision of feedback at scale](https://reader036.vdocuments.mx/reader036/viewer/2022062503/58f9b7cc1a28ab0c238b45cf/html5/thumbnails/27.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 27
Helpful feedback
Effect size (Cohen’s d) = 0.49. Medium positive effect
Midterm Scores
Effect size (Cohen’s d) = 0.21. Small positive effect
Pardo, A., Jovanovic, J., Dawson, S., Gašević, D. & Mirriahi, N. (2017). Using Learning Analytics to Scale the Provision of Personalised Feedback. Manuscript in preparation
![Page 28: The role of data in the provision of feedback at scale](https://reader036.vdocuments.mx/reader036/viewer/2022062503/58f9b7cc1a28ab0c238b45cf/html5/thumbnails/28.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 28
What if I tell you…
![Page 29: The role of data in the provision of feedback at scale](https://reader036.vdocuments.mx/reader036/viewer/2022062503/58f9b7cc1a28ab0c238b45cf/html5/thumbnails/29.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 29
ontasklearning.org
![Page 30: The role of data in the provision of feedback at scale](https://reader036.vdocuments.mx/reader036/viewer/2022062503/58f9b7cc1a28ab0c238b45cf/html5/thumbnails/30.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 30
IF [video 3.9.5 not watched] THEN “…."
ontasklearning.org
![Page 31: The role of data in the provision of feedback at scale](https://reader036.vdocuments.mx/reader036/viewer/2022062503/58f9b7cc1a28ab0c238b45cf/html5/thumbnails/31.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 31
• Support instructors to create personalised feedback
• Simple rule-base knowledge encoding
• Provide appropriate view of data sources
• Scale to large and highly diverse cohorts
• Will be released as open-source project Q3/4 2017
• First pilots in Q1/2 2017
• Tutorial in LAK 2017
• Contact us if interested
ontasklearning.org
![Page 32: The role of data in the provision of feedback at scale](https://reader036.vdocuments.mx/reader036/viewer/2022062503/58f9b7cc1a28ab0c238b45cf/html5/thumbnails/32.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 32
Bill Tyne flickr.com
• Data to understand and improve learning needs to be situated and connected to actions
• Feedback is a powerful focal point to drive data-supported decisions
• Explore the design of theory-driven learning analytics tools that connect data with support actions
![Page 33: The role of data in the provision of feedback at scale](https://reader036.vdocuments.mx/reader036/viewer/2022062503/58f9b7cc1a28ab0c238b45cf/html5/thumbnails/33.jpg)
Abelardo Pardo (@abelardopardo)Faculty of Engineering and IT
slideshare.net/abelardo_pardo
The role of data in the provision of feedback at scale
Wally G
obetz flickr.com
ASCILITE Webinar 11 April 2017