assessing student self-assessment: an additional argument for blended learning - fr arico
TRANSCRIPT
![Page 1: Assessing Student Self-Assessment: An Additional Argument for Blended Learning - FR Arico](https://reader033.vdocuments.mx/reader033/viewer/2022051516/55a414611a28abd41a8b479e/html5/thumbnails/1.jpg)
1
Assessing Student Self-Assessment:
An Additional Argument for
Blended Learning
Fabio R. Aricò
Duncan Watson
Chris Thomson
NRHE SRHE – Dec 2014
![Page 2: Assessing Student Self-Assessment: An Additional Argument for Blended Learning - FR Arico](https://reader033.vdocuments.mx/reader033/viewer/2022051516/55a414611a28abd41a8b479e/html5/thumbnails/2.jpg)
ACKNOWLEDGEMENTS
UEA-HEFCE Widening Participation Teaching Fellowship
HEA – Teaching Development Grant Scheme
2
![Page 3: Assessing Student Self-Assessment: An Additional Argument for Blended Learning - FR Arico](https://reader033.vdocuments.mx/reader033/viewer/2022051516/55a414611a28abd41a8b479e/html5/thumbnails/3.jpg)
OUTLINE
1. Overview of the “When Student Confidence Clicks Project”
2. Introduction to 2 learning environments to be compared:seminars and workshops
Learning analytics:the relationship between attainment & self-assessment
4. Final remarks: assessing self-assessment
5. Further research
3
![Page 4: Assessing Student Self-Assessment: An Additional Argument for Blended Learning - FR Arico](https://reader033.vdocuments.mx/reader033/viewer/2022051516/55a414611a28abd41a8b479e/html5/thumbnails/4.jpg)
ETHICAL REMARK
You will be presented with data collected during teaching sessions.
Students involved have given informed consent for me to analyse their responses and present the results of this analysis.
I can assist with ethical queries as well, please ask me.
4
![Page 5: Assessing Student Self-Assessment: An Additional Argument for Blended Learning - FR Arico](https://reader033.vdocuments.mx/reader033/viewer/2022051516/55a414611a28abd41a8b479e/html5/thumbnails/5.jpg)
1. When Student Confidence Clicks: the role of Academic Self-Efficacy in Learning
5
![Page 6: Assessing Student Self-Assessment: An Additional Argument for Blended Learning - FR Arico](https://reader033.vdocuments.mx/reader033/viewer/2022051516/55a414611a28abd41a8b479e/html5/thumbnails/6.jpg)
PROJECT DEVELOPMENTS
Project Resources: https://sites.google.com/site/fabioarico/hea_tdg
Project Workshop: Wednesday 3 Sep 2014 - University of East Anglia
6
![Page 7: Assessing Student Self-Assessment: An Additional Argument for Blended Learning - FR Arico](https://reader033.vdocuments.mx/reader033/viewer/2022051516/55a414611a28abd41a8b479e/html5/thumbnails/7.jpg)
BLENDED LEARNING ENVIRONMENT
• Intense use of Student Response Systems (clickers).
• Focus on increasing students’ Academic Self-Efficacy beliefs Self-Assessment is a fundamental component.
• Intense use of SRS + VLE dialogue with the students – closing feedback loop blended learning, but also blended surveying!
• Learning analytics to disentangle relationship between different aspects of the learning experience.
7
![Page 8: Assessing Student Self-Assessment: An Additional Argument for Blended Learning - FR Arico](https://reader033.vdocuments.mx/reader033/viewer/2022051516/55a414611a28abd41a8b479e/html5/thumbnails/8.jpg)
DATASETS
Student Q1 Q2 Q3 …
1 0 1 1
2 1 0 0
3 1 1 …
…
performance per questionconfidence by question
pe
rform
ance
pe
r stud
en
tco
nfid
en
ce by stu
de
nt
8
longitudinal study- across all lectures- across all seminars- across all workshops
Intermediate and final attainment outcomes- course test- final exam
![Page 9: Assessing Student Self-Assessment: An Additional Argument for Blended Learning - FR Arico](https://reader033.vdocuments.mx/reader033/viewer/2022051516/55a414611a28abd41a8b479e/html5/thumbnails/9.jpg)
EVIDENCE-BASED PRACTICE
9
ATTAINMENT
ENGAGEMENT CONFIDENCE
Gender
Satisfaction
Previous attainmentDomicilePOLAR
Perceived difficulties
![Page 10: Assessing Student Self-Assessment: An Additional Argument for Blended Learning - FR Arico](https://reader033.vdocuments.mx/reader033/viewer/2022051516/55a414611a28abd41a8b479e/html5/thumbnails/10.jpg)
2. Workshops and Seminars
Attainment & Self-Assessment
10
![Page 11: Assessing Student Self-Assessment: An Additional Argument for Blended Learning - FR Arico](https://reader033.vdocuments.mx/reader033/viewer/2022051516/55a414611a28abd41a8b479e/html5/thumbnails/11.jpg)
MOTIVATION and CORE IDEAS
• This is a by-product of my HEA TDG project, which is taking life on its own Self-Assessment as an important determinant of Self-Efficacy beliefs
(Bandura, 1977 – Pajares, 1996)
• Self-Assessment skills contribute to form autonomous students HE learning and teaching practice should develop these
(Boud, 1995 – Boud and Falchikov, 2006/7 – Evans, 2013)
• Empirical evidence seems to identify positive association between self-assessment skills and performance in academic settings Dunning et.al. (2003), Zell & Krizan (2014)
• Challenge: does (self)assessment design have a role to play?
11
![Page 12: Assessing Student Self-Assessment: An Additional Argument for Blended Learning - FR Arico](https://reader033.vdocuments.mx/reader033/viewer/2022051516/55a414611a28abd41a8b479e/html5/thumbnails/12.jpg)
TEACHING PROTOCOL – the module
Introductory Macroeconomics Level 1 – compulsory year-long module - 170 students
Lectures traditional frontal-teaching (10 per sem.)
Seminars small group, pre-assigned problem sets (4 per sem.)
Workshops large group, problem-solving sessions (4 per sem.)
Support Sessions non-compulsory drop-in sessions (4 per sem.)
12
![Page 13: Assessing Student Self-Assessment: An Additional Argument for Blended Learning - FR Arico](https://reader033.vdocuments.mx/reader033/viewer/2022051516/55a414611a28abd41a8b479e/html5/thumbnails/13.jpg)
WORKSHOPS – learning environment
• Large flipped-class environment (2 Groups - 85 students each)
• 1st part: Multiple choice questions (4 choices) - 2nd part: problem-solving
• Iterated algorithm:
1. Ask Question 1st round 4 choices (distribution not revealed)
2. Ask Self-Assessment Question 4 levels Likert-scale (strongly/agree – strongly/disagree)
3. Peer-Instruction
4. Ask Question 2nd round show answers – teach correct answer and give explanations.
13
![Page 14: Assessing Student Self-Assessment: An Additional Argument for Blended Learning - FR Arico](https://reader033.vdocuments.mx/reader033/viewer/2022051516/55a414611a28abd41a8b479e/html5/thumbnails/14.jpg)
WORKSHOPS – data coding
For each session (7 in a year):
• Code 1st response: 1 = correct 0 = incorrect
• Code confidence in response: 1 = strongly/agree0 = strongly/disagree
• Compute average score and average confidence per student.
attainment self-assessmentmeasure measure
14
![Page 15: Assessing Student Self-Assessment: An Additional Argument for Blended Learning - FR Arico](https://reader033.vdocuments.mx/reader033/viewer/2022051516/55a414611a28abd41a8b479e/html5/thumbnails/15.jpg)
WORKSHOPS – data coding
For each student (in each session):
If student average score > session average score high-attainmentotherwise low-attainment
METHOD A:
If student average conf. > session average conf. high-confidenceotherwise low-confidence
METHOD B:
If student average conf. > 0.5 high-confidenceotherwise low-confidence
15
![Page 16: Assessing Student Self-Assessment: An Additional Argument for Blended Learning - FR Arico](https://reader033.vdocuments.mx/reader033/viewer/2022051516/55a414611a28abd41a8b479e/html5/thumbnails/16.jpg)
WORKSHOPS – data analysis
Cross-tabulate results:
16
![Page 17: Assessing Student Self-Assessment: An Additional Argument for Blended Learning - FR Arico](https://reader033.vdocuments.mx/reader033/viewer/2022051516/55a414611a28abd41a8b479e/html5/thumbnails/17.jpg)
WORKSHOPS - results
What is the relationship between attainment and confidence?
17
![Page 18: Assessing Student Self-Assessment: An Additional Argument for Blended Learning - FR Arico](https://reader033.vdocuments.mx/reader033/viewer/2022051516/55a414611a28abd41a8b479e/html5/thumbnails/18.jpg)
WORKSHOP - results
What is the relationship between attainment and confidence?
18
![Page 19: Assessing Student Self-Assessment: An Additional Argument for Blended Learning - FR Arico](https://reader033.vdocuments.mx/reader033/viewer/2022051516/55a414611a28abd41a8b479e/html5/thumbnails/19.jpg)
SEMINARS – learning environment
• Small-class environment (Groups of 13-15 students each)
• 1st part: Multiple choice quiz (4 choices) - 2nd part: problem-solving
• Structure of the Quiz:
- Paper based
- 5-6 minutes duration
- 3 or 4 formative questions
- 1 self-assessment question on the quiz
- 1 self-efficacy question about future beliefs.
19
![Page 20: Assessing Student Self-Assessment: An Additional Argument for Blended Learning - FR Arico](https://reader033.vdocuments.mx/reader033/viewer/2022051516/55a414611a28abd41a8b479e/html5/thumbnails/20.jpg)
SEMINARS – data coding
For each session (7 in a year):
• Code each answer: 1 = correct 0 = incorrect
• Code confidence in response: 1 = strongly/agree0 = strongly/disagree
• Compute average score attainment measure
There is just one confidence response confidence measure
20
![Page 21: Assessing Student Self-Assessment: An Additional Argument for Blended Learning - FR Arico](https://reader033.vdocuments.mx/reader033/viewer/2022051516/55a414611a28abd41a8b479e/html5/thumbnails/21.jpg)
SEMINARS – data coding
For each student (in each session):
If student average score > session average score high-attainmentotherwise low-attainment
If student confidence = 1 high-confidenceotherwise low-confidence
Note the asymmetry with workshops.This was addressed this year, asking about confidence question by question.
21
![Page 22: Assessing Student Self-Assessment: An Additional Argument for Blended Learning - FR Arico](https://reader033.vdocuments.mx/reader033/viewer/2022051516/55a414611a28abd41a8b479e/html5/thumbnails/22.jpg)
SEMINARS - results
What is the relationship between attainment and confidence?
22
![Page 23: Assessing Student Self-Assessment: An Additional Argument for Blended Learning - FR Arico](https://reader033.vdocuments.mx/reader033/viewer/2022051516/55a414611a28abd41a8b479e/html5/thumbnails/23.jpg)
SEMINARS - results
What is the relationship between attainment and confidence?
23
![Page 24: Assessing Student Self-Assessment: An Additional Argument for Blended Learning - FR Arico](https://reader033.vdocuments.mx/reader033/viewer/2022051516/55a414611a28abd41a8b479e/html5/thumbnails/24.jpg)
3. Final Remarks
Assessing Self-Assessment
24
![Page 25: Assessing Student Self-Assessment: An Additional Argument for Blended Learning - FR Arico](https://reader033.vdocuments.mx/reader033/viewer/2022051516/55a414611a28abd41a8b479e/html5/thumbnails/25.jpg)
PREVIOUS LITERATURE
• In literature there is a prevailing idea:
low-attainment students are not able to self-assess their performance.
• Dunning, D., Johnson, K., Ehrlinger, J., and Kruger, J., (2003), “Why People Fail
to Recognise their Own Incompetence”, Current Directions in Psychological
Science, 12:83, 83-87.
• But the results find in my research seem to question the learning
environment within which self-assessment is being observed.
the learning environment affects the relationship attainment-confidence.
25
![Page 26: Assessing Student Self-Assessment: An Additional Argument for Blended Learning - FR Arico](https://reader033.vdocuments.mx/reader033/viewer/2022051516/55a414611a28abd41a8b479e/html5/thumbnails/26.jpg)
OVERALL RESULTS
• In Seminar Quizzes:
high-attainment students display higher confidence
low-attainment students not able to self-assess their performance.
• In Workshop sessions:
high-attainment students display higher confidence
low-attainment students display lower confidence.
• How to interpret this asymmetry?
26
![Page 27: Assessing Student Self-Assessment: An Additional Argument for Blended Learning - FR Arico](https://reader033.vdocuments.mx/reader033/viewer/2022051516/55a414611a28abd41a8b479e/html5/thumbnails/27.jpg)
OVERALL RESULTS
• In Seminar Quizzes:
3 or 4 questions, paper-based quiz, 5-6 minutes, not anonymous, small class
1 confidence assessment for overall performance.
• In Workshop sessions:
5-10 questions, clicker response, slower pace, quasi-anonymous, large class
1 confidence assessment for each question asked.
27
![Page 28: Assessing Student Self-Assessment: An Additional Argument for Blended Learning - FR Arico](https://reader033.vdocuments.mx/reader033/viewer/2022051516/55a414611a28abd41a8b479e/html5/thumbnails/28.jpg)
OVERALL RESULTS
• Low-attainment students encounter more difficulties in
self-assessing their performance in an environment where:
they self-assess their ‘overall’ performance on a composite task
they are exposed to questions for a shorter period of time
they are exposed to fewer questions, not anonymously.
• Focus group interviews (differentiated by attainment groups)
confirm that low-attainment students display poorer self-assessment skills.
• So what is happening in workshops?
28
![Page 29: Assessing Student Self-Assessment: An Additional Argument for Blended Learning - FR Arico](https://reader033.vdocuments.mx/reader033/viewer/2022051516/55a414611a28abd41a8b479e/html5/thumbnails/29.jpg)
4. Further Research
29
![Page 30: Assessing Student Self-Assessment: An Additional Argument for Blended Learning - FR Arico](https://reader033.vdocuments.mx/reader033/viewer/2022051516/55a414611a28abd41a8b479e/html5/thumbnails/30.jpg)
FURTHER RESEARCH
• Introduce confidence evaluation question-by-question in Seminars.
• Explore the relationship between objective and subjective evaluation
relationship between response entropy and student confidence.
• “Peer-Instruction Unveiled”
relationship between 1st and 2nd response – ‘learning gains’
relationship between learning gains and student confidence.
• Learning analytics at student-level
investigate the role of demographics
investigate the impact of formative assessment on summative assessment.
30