the role of data in the provision of feedback at scale
TRANSCRIPT
![Page 1: The role of data in the provision of feedback at scale](https://reader034.vdocuments.mx/reader034/viewer/2022051520/5883040f1a28abe70d8b60e3/html5/thumbnails/1.jpg)
Abelardo Pardo (@abelardopardo)Faculty of Engineering and IT slideshare.net/abelardo_pardo
The role of data in the provision of feedback at scale
APRU ERT Forum 18 November 2016
Faruk Ateş flickr.com
![Page 2: The role of data in the provision of feedback at scale](https://reader034.vdocuments.mx/reader034/viewer/2022051520/5883040f1a28abe70d8b60e3/html5/thumbnails/2.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 2
It’s the learning!
Richard Katz presentation APRU ERT Forum Presentation 16 Nov 2016, Singapore
![Page 3: The role of data in the provision of feedback at scale](https://reader034.vdocuments.mx/reader034/viewer/2022051520/5883040f1a28abe70d8b60e3/html5/thumbnails/3.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 3
![Page 4: The role of data in the provision of feedback at scale](https://reader034.vdocuments.mx/reader034/viewer/2022051520/5883040f1a28abe70d8b60e3/html5/thumbnails/4.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 4
Tim Pierce flickr.com
![Page 5: The role of data in the provision of feedback at scale](https://reader034.vdocuments.mx/reader034/viewer/2022051520/5883040f1a28abe70d8b60e3/html5/thumbnails/5.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 5
Follow a scholarly approachJonathan C
ohen flickr.com
![Page 6: The role of data in the provision of feedback at scale](https://reader034.vdocuments.mx/reader034/viewer/2022051520/5883040f1a28abe70d8b60e3/html5/thumbnails/6.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 6
Yvonne Larsson flickr.comPressure Quality/Cost
Student Feedback
Data-driven Support
Examples
![Page 7: The role of data in the provision of feedback at scale](https://reader034.vdocuments.mx/reader034/viewer/2022051520/5883040f1a28abe70d8b60e3/html5/thumbnails/7.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 7
Yvonne Larsson flickr.comPressureQuality/Cost
![Page 8: The role of data in the provision of feedback at scale](https://reader034.vdocuments.mx/reader034/viewer/2022051520/5883040f1a28abe70d8b60e3/html5/thumbnails/8.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 8
Space & Light flickr.com
Competition
Funding
Enrolment
Expectations
Digital Environments
![Page 9: The role of data in the provision of feedback at scale](https://reader034.vdocuments.mx/reader034/viewer/2022051520/5883040f1a28abe70d8b60e3/html5/thumbnails/9.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 9
Tulane Public Relations flickr.com
Instructors know how to support students
but not in the current conditions
![Page 10: The role of data in the provision of feedback at scale](https://reader034.vdocuments.mx/reader034/viewer/2022051520/5883040f1a28abe70d8b60e3/html5/thumbnails/10.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 10
Focus on "good instruction"D
aniel Kwok flickr.com
Kirschner, P. A., & Neelen, M. (2016). The cold left-overs of inquiry-based learning. Retrieved from https://3starlearningexperiences.wordpress.com/2016/11/15/the-cold-left-overs-of-inquiry-based-learning/
![Page 11: The role of data in the provision of feedback at scale](https://reader034.vdocuments.mx/reader034/viewer/2022051520/5883040f1a28abe70d8b60e3/html5/thumbnails/11.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 11
increased engagement = increase in quality of experience
Matthias Ripp flickr.com
![Page 12: The role of data in the provision of feedback at scale](https://reader034.vdocuments.mx/reader034/viewer/2022051520/5883040f1a28abe70d8b60e3/html5/thumbnails/12.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 12
Startup Mena flickr.com
Increase in feedback = increase in engagement
![Page 13: The role of data in the provision of feedback at scale](https://reader034.vdocuments.mx/reader034/viewer/2022051520/5883040f1a28abe70d8b60e3/html5/thumbnails/13.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 13
Yvonne Larsson flickr.comPressure Quality/Cost
StudentFeedback
![Page 14: The role of data in the provision of feedback at scale](https://reader034.vdocuments.mx/reader034/viewer/2022051520/5883040f1a28abe70d8b60e3/html5/thumbnails/14.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale
If you could choose one…• More than 500 meta-analyses
of student achievements
• 100 factors with potential influence
• Feedback in top five
• (74 meta-analyses) Most effective form: video, audio, computer-assisted instructional feedback, and/or related goals
14Hattie, J. A. (1999). Influences on student learning. Inaugural professorial address, University of Auckland, New Zealand
![Page 15: The role of data in the provision of feedback at scale](https://reader034.vdocuments.mx/reader034/viewer/2022051520/5883040f1a28abe70d8b60e3/html5/thumbnails/15.jpg)
Abelardo Pardo Using technology to provide feedback on learning strategies
The feedback question gets systematically lower values in student surveys
15
Krause, K.-L., Hartley, R., James, R., & McInnis, C. (2005). The First Year Experience in Australian Universities: Findings from a decade of National Studies. University of Melbourne: Centre for the Study of Higher Education.
Eleaf flickr.com
![Page 16: The role of data in the provision of feedback at scale](https://reader034.vdocuments.mx/reader034/viewer/2022051520/5883040f1a28abe70d8b60e3/html5/thumbnails/16.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 16
Hattie, J., & Timperley, H. (2007). The Power of Feedback. Review of Educational Research, 77(1), 81-112. doi:10.3102/003465430298487
Faruk Ateş flickr.com
"information provided by an agent (teacher, peer, book, parent, self, experience) regarding aspects
of one’s performance or understanding”
![Page 17: The role of data in the provision of feedback at scale](https://reader034.vdocuments.mx/reader034/viewer/2022051520/5883040f1a28abe70d8b60e3/html5/thumbnails/17.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 17
Hounsell, D. (2007). Toward more sustainable feedback to students. In D. Boud & N. Falchikov (Eds.), Rethinking Assessment in Higher Education: Learning for the Longer Term. London and New York: Routledge.
Perceived as an administrative chore instead of a pedagogical necessity
Marcin W
ichary flickr.com
![Page 18: The role of data in the provision of feedback at scale](https://reader034.vdocuments.mx/reader034/viewer/2022051520/5883040f1a28abe70d8b60e3/html5/thumbnails/18.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 18
Allen Lai flickr.com
Hounsell, D. (2007). Toward more sustainable feedback to students. In D. Boud & N. Falchikov (Eds.), Rethinking Assessment in Higher Education: Learning for the Longer Term. London and New York: Routledge.
Downward spiral: Student disenchantment mounts when feedback is uninformative and still,
less feedback is provided
Solution: provide high-value feedback, rethink the role of student, enhance relation between guidance and feedback
![Page 19: The role of data in the provision of feedback at scale](https://reader034.vdocuments.mx/reader034/viewer/2022051520/5883040f1a28abe70d8b60e3/html5/thumbnails/19.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 19
Hattie, J., & Timperley, H. (2007). The Power of Feedback. Review of Educational Research, 77(1), 81-112. doi:10.3102/003465430298487
Orkom
edix flickr.com
Feedback Levels1. Task Level (understanding,
performance)
2. Process Level (what to do to understand, perform)
3. Self-regulation level (detecting and directing effort)
4. Self level (personal evaluation and affect)
![Page 20: The role of data in the provision of feedback at scale](https://reader034.vdocuments.mx/reader034/viewer/2022051520/5883040f1a28abe70d8b60e3/html5/thumbnails/20.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 20
Bjork, R. A., Dunlosky, J., & Kornell, N. (2013). Self-regulated learning: beliefs, techniques, and illusions. Annu Rev Psychol, 64, 417-444. doi:10.1146/annurev-psych-113011-143823
• May not know how to promote comprehension, retention, transfer.
• May not assess properly our own learning
• May be biased when judging our learning
• May rely too much on social beliefs
• Towards “adaptive learners”
Christian W
eidinger flickr.com
![Page 21: The role of data in the provision of feedback at scale](https://reader034.vdocuments.mx/reader034/viewer/2022051520/5883040f1a28abe70d8b60e3/html5/thumbnails/21.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 21
Janne Moren flickr.com
Dunlosky, J. (2013). Strengthening the Student Toolbox. Study Strategies to Boost Learning. American Educator, 37(3), 12-21.
Suggest learning strategies to students
![Page 22: The role of data in the provision of feedback at scale](https://reader034.vdocuments.mx/reader034/viewer/2022051520/5883040f1a28abe70d8b60e3/html5/thumbnails/22.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 22
Personalised suggestions
Woody1778a flickr.com
![Page 23: The role of data in the provision of feedback at scale](https://reader034.vdocuments.mx/reader034/viewer/2022051520/5883040f1a28abe70d8b60e3/html5/thumbnails/23.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 23
Laverrue flickr.comIncrease in personalisation = increase in knowledge
![Page 24: The role of data in the provision of feedback at scale](https://reader034.vdocuments.mx/reader034/viewer/2022051520/5883040f1a28abe70d8b60e3/html5/thumbnails/24.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 24
From visualisations to actions
Pardo, A., Mirriahi, N., Martínez-Maldonado, R., Jovanović, J., Dawson, S., & Gasevic, D. (2016). Generating Actionable Predictive Models of Academic Performance. Paper presented at the International Conference on Learning Analytics and Knowledge, Edinburgh.
Kannan B flickr.com
![Page 25: The role of data in the provision of feedback at scale](https://reader034.vdocuments.mx/reader034/viewer/2022051520/5883040f1a28abe70d8b60e3/html5/thumbnails/25.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 25
Yvonne Larsson flickr.com
Pressure Quality/Cost
Student Feedback
Data-drivenSupport
![Page 26: The role of data in the provision of feedback at scale](https://reader034.vdocuments.mx/reader034/viewer/2022051520/5883040f1a28abe70d8b60e3/html5/thumbnails/26.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 26
PNN
L Pacific Northw
est flickr.com
• Collect data about how students participate in a learning experience
• Translate observations into actions
• Take into account the instructional design
• Expertise of the instructor
• Create and deploy personalised support actions
![Page 27: The role of data in the provision of feedback at scale](https://reader034.vdocuments.mx/reader034/viewer/2022051520/5883040f1a28abe70d8b60e3/html5/thumbnails/27.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 27
Intan A3 flickr.com1) Go beyond feedback at the task level
![Page 28: The role of data in the provision of feedback at scale](https://reader034.vdocuments.mx/reader034/viewer/2022051520/5883040f1a28abe70d8b60e3/html5/thumbnails/28.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 28
2) Comprehensive*, multimodal, real-time data collection
bodhithaj flickr.com
![Page 29: The role of data in the provision of feedback at scale](https://reader034.vdocuments.mx/reader034/viewer/2022051520/5883040f1a28abe70d8b60e3/html5/thumbnails/29.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 29
3) Clean, relevant data sources at the instructor’s fingertipsN
ick Readhead flickr.com
![Page 30: The role of data in the provision of feedback at scale](https://reader034.vdocuments.mx/reader034/viewer/2022051520/5883040f1a28abe70d8b60e3/html5/thumbnails/30.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 30
Small Realm
flickr.com
4) Capacity to deploy immediate personalised support actions
![Page 31: The role of data in the provision of feedback at scale](https://reader034.vdocuments.mx/reader034/viewer/2022051520/5883040f1a28abe70d8b60e3/html5/thumbnails/31.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 31
IAN RAN
SLEY DESIG
N PLU
S ILLUSTRATIO
N flickr.com
5) Scale to large or highly diverse student cohorts
![Page 32: The role of data in the provision of feedback at scale](https://reader034.vdocuments.mx/reader034/viewer/2022051520/5883040f1a28abe70d8b60e3/html5/thumbnails/32.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 32
Yvonne Larsson flickr.com
Pressure Quality/Cost
Student Feedback
Data-driven Support
Examples
![Page 33: The role of data in the provision of feedback at scale](https://reader034.vdocuments.mx/reader034/viewer/2022051520/5883040f1a28abe70d8b60e3/html5/thumbnails/33.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 33
Tracking
![Page 34: The role of data in the provision of feedback at scale](https://reader034.vdocuments.mx/reader034/viewer/2022051520/5883040f1a28abe70d8b60e3/html5/thumbnails/34.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 34
You should take a more careful look at how symbols are encoded in the video. Would you be able to encode/decode UAL symbols without looking at the video?
Good initial work. However, did you understand the trick to handle encoding with a variable number of bits? Would you be able to provide an example?
Good work. Would you be able to come up with your own machine language and your encoding scheme? Remember that it has to be unambiguous.
Thorough work with the task about machine language encoding. Give it a quick review before the midterm.
Q1 Q2 Q3 Q4
Instructor
![Page 35: The role of data in the provision of feedback at scale](https://reader034.vdocuments.mx/reader034/viewer/2022051520/5883040f1a28abe70d8b60e3/html5/thumbnails/35.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 35
AutomaticEmail
![Page 36: The role of data in the provision of feedback at scale](https://reader034.vdocuments.mx/reader034/viewer/2022051520/5883040f1a28abe70d8b60e3/html5/thumbnails/36.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 36
1) Email remains the primary and preferred channel of communication with the University
http://libinnovation.blogspot.sg/2016/11/how-do-students-really-prefer-to.html
![Page 37: The role of data in the provision of feedback at scale](https://reader034.vdocuments.mx/reader034/viewer/2022051520/5883040f1a28abe70d8b60e3/html5/thumbnails/37.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 37
Effect size (Cohen’s d) = 0.49. Medium positive effect
Effect size (Cohen’s d) = 0.21. Small positive effect
![Page 38: The role of data in the provision of feedback at scale](https://reader034.vdocuments.mx/reader034/viewer/2022051520/5883040f1a28abe70d8b60e3/html5/thumbnails/38.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 38
Liu, D. Y.-T., Bartimote-Aufflick, K., Pardo, A., & Bridgeman, A. J. (2016). Data-driven Personalization of Student Learning Support in Higher Education. In A. Peña-Ayala (Ed.), Learning analytics: Fundaments, applications, and trends: A view of the current state of the art. In preparation: Springer.
Instructors define simple rules to create personalised emails from multiple data sources
Student Relationship Engagement System
Not so m
uch flickr.com
![Page 39: The role of data in the provision of feedback at scale](https://reader034.vdocuments.mx/reader034/viewer/2022051520/5883040f1a28abe70d8b60e3/html5/thumbnails/39.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 39
ontasklearning.org (in production)
IF [video 3.9.5 not watched] THEN “…."
![Page 40: The role of data in the provision of feedback at scale](https://reader034.vdocuments.mx/reader034/viewer/2022051520/5883040f1a28abe70d8b60e3/html5/thumbnails/40.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 40
Alan Levine flickr.com
Workshop to
• Expose academics and educational designers to data sets
• Frame the need for personalised support
• Explore immediate vs long-term actions
• Jumpstart the conversation
• Identify institutional needs
![Page 41: The role of data in the provision of feedback at scale](https://reader034.vdocuments.mx/reader034/viewer/2022051520/5883040f1a28abe70d8b60e3/html5/thumbnails/41.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 41
Pardo, A., Jovanović, J., Dawson, S., Gasevic, D., & Mirriahi, N. (2016). Exploring student interactions with preparation activities in a flipped learning experience. Computers & Education, Manuscript under review.
Identify student interactions in flipped learning environment
Yvonne Larsson flickr.com
![Page 42: The role of data in the provision of feedback at scale](https://reader034.vdocuments.mx/reader034/viewer/2022051520/5883040f1a28abe70d8b60e3/html5/thumbnails/42.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 42
Jovanović, J., Gasevic, D., Pardo, A., Dawson, S., & Mirriahi, N. (2016). Learning Analytics to Unveil Learning Strategies in a Flipped Classroom. The Internet and Higher Education, Manuscript under review.
1. identify engagement patterns in active learning environments
2. Ground for personalised suggestions 3. Increase student awareness about strategies
Luc Legay flickr.com
![Page 43: The role of data in the provision of feedback at scale](https://reader034.vdocuments.mx/reader034/viewer/2022051520/5883040f1a28abe70d8b60e3/html5/thumbnails/43.jpg)
Abelardo Pardo The role of data in the provision of feedback at scale 43
Ed Dunens flickr.com
Focus on improvements on the learning experienceData provides insight on engagementCombine data with instructor expertise
Use technology to scale personalisation
![Page 44: The role of data in the provision of feedback at scale](https://reader034.vdocuments.mx/reader034/viewer/2022051520/5883040f1a28abe70d8b60e3/html5/thumbnails/44.jpg)
Abelardo Pardo (@abelardopardo)Faculty of Engineering and IT slideshare.net/abelardo_pardo
The role of data in the provision of feedback at scale
APRU ERT Forum 18 November 2016
Faruk Ateş flickr.com