an empirical study of in-class labs on student learning of linear data structures sarah heckman...
TRANSCRIPT
An Empirical Study of In-Class Labs on Student Learning of Linear Data StructuresSarah HeckmanTeaching Associate ProfessorDepartment of Computer ScienceNorth Carolina State University
ICER 2015
Problem
ICER 2015
CSC116 CSC216 CSC316
• 7-8 sections• 33 students• 1 instructor• 2 TAs• Lecture/Lab
• 1-2 sections• 70-90 students• 1 instructor• 2-3 TAs• Lecture
• 1-2 sections• 70-90 students• 1 instructor• 2-3 TAs• Lecture
Transition!
Retention!
Lab? In-Class Labs?Do Nothing?
Research Goal
• To increase student learning and engagement through in-class laboratories on linear data structures
• Hypothesis: active learning practices that involve larger problems would increase student learning and engagement
ICER 2015
In-class Labs > Pair & Share
Research Questions
• Do in-class laboratories on linear data structures increase student learning on linear data structures exam questions when compared to active-learning lectures?
• Do in-class laboratories on linear data structures increase student engagement on linear data structures exam questions when compared to active-learning lectures?
ICER 2015
Active Learning in CSC216• “engaging the students in the process of learning
through activities and/or discussion in class, as opposed to passively listening to an expert” [Freeman, et al. 2014]
• Control: Active Learning Lectures – 2-5 pair & share exercises per class– Submitted through Google forms
• Treatment: In-class Labs– Lab activity for the entire lecture period– Pre-class videos introduced topic
ICER 2015
Study Participants
Metric Section 001 Section 002# Enrolled 85 102Participants (completed course)
49 60
Dropped/Withdrawn (consenting only)
3 4
Women 9 10Meeting Time TH 2:20-3:35p MW 2:20-3:35p
ICER 2015
• Self-selected into section during standard registration period• Populations were similar as measured by a survey on
experience with tooling and self-efficacy.
Methods• Quasi-Experimental
– Counter-balanced design– Learning measured through exams– Engagement measured through observations of class
meetings
ICER 2015
Lists
Array Array
Array Array Linked Linked
Linked Linked
Iterators
Exam 1
001
002
Replication Materials:http://people.engr.ncsu.edu/sesmith5/216-labs/csc216_labs.html
Replication Materials:http://people.engr.ncsu.edu/sesmith5/216-labs/csc216_labs.html
Observed Class Meetings
Student Learning – Exam 1
• Part 4: Method Tracing with ArrayLists• Part 5: Writing an ArrayList method
ICER 2015
Item Points S001 Mean
S001 SD
S002 Mean
S002 SD
p-value
E1 P4#8 5 3.63 1.56 4.35 1.45 < 0.010
E1 P4#9 5 4.18 1.07 4.57 1.09 0.016
E1 P4#10 5 2.63 2.40 3.45 2.18 0.149
E1 P4 15 10.45 3.97 12.37 3.74 < 0.010
E1 P5 20 17.76 4.0 18.25 4.09 0.233
Student Learning – Exam 2
• Part 3 – Linked Node Transformation• Part 5 – Writing a LinkedList Method
ICER 2015
Item Points S001 Mean
S001 SD
S002 Mean
S002 SD
p-value
E2 P3 16 9.43 5.85 11.80 6.41 < 0.010
E2 P5 20 11.80 4.14 12.58 4.21 0.412
Student Learning – Exam 3
• Comprehensive 3 hour final exam• Stack Using an ArrayList• Queue Using a LinkedList
ICER 2015
Item Points S001 Mean
S001 SD
S002 Mean
S002 SD
p-value
E3 Array 10 8.31 2.45 8.46 2.49 0.313
E3 Linked 10 8.36 2.53 881 2.45 0.221
E3 Score 105 85.02 29.17 87.23 28.92 0.372
Student Engagement
• Observations for ArrayList and LinkedList class meetings
• Observers were graduate students and a colleague participating in a Teaching and Learning seminar
• Counts of students off topic during lecture and exercise portions of the class
• Some inconsistent use of the observation protocol
ICER 2015
Student Engagement
ICER 2015
Observation Class Type
# Off Topic – Lecture
# Off Topic – Exercise
Questions of Teaching Staff
1 Lab 5 7 32
2 Lecture 62 49 12
3 Lab 10 43 50
4 Lecture 46 16 ---
5 Lecture --- --- ---
6 Lab 5 10 33
7 Lecture 52 54 2
8 Lab 16 5 ---
Lab Average 9 16.3 38.3
Lecture Average 53.3 39.7 7
Lecture / Lab 5.9 2.4 0.2
Threats to Validity
• External Validity– Two sections of the same course, taught by the same
instructor, in the same semester, and same time of day
– Replication needed in other contexts to generalize further
– Could provide additional data points in future meta-analyses
ICER 2015
Threats to Validity• Internal Validity
– Selection bias: students selected their own sections• Initial surveys shows groups were similar
– Confounding factors• Materials shared between groups• Effect size – only 6 in-class labs
– Differential Attrition Bias• Considered “soft-drops” in the study
– Experimenter Bias• Participants were not revealed until after the semester was over
ICER 2015
Threats to Validity
• Construct Validity– Exams as Measures of Learning
• Exam 1 and Exam 2 were similar, but not the same, between sections
• Exam 3 was common• Does exam really measure student learning?
– Survey• Wording may be confusing for prior tool experience• Efficacy questions not a validated instrument
– Observation Protocol as Measure of Engagement• Inconsistent use by observers
ICER 2015
Discussion• Did in-class labs increase student learning?
– No, at least not as measured by exam questions– Both control and intervention were active learning
• Maybe a simple active learning intervention is enough– Comparisons with earlier semesters may show more
• Did in-class labs increase student engagement?– Yes and No– The atmosphere in the classroom was fantastic– But many questions were technology and not concept
• Completion – 72% of students earned a C or higher– Not reaching the higher levels of completion we expect from active
learning literature
ICER 2015
Future Work• Additional Work on Fall 2014 Data
– Compare results on final exam with previous courses– Incorporate analysis of other measures of learning –
projects, exercises, etc.• Starting in Fall 2015
– Additional in-class labs → Lab-based course– Measure types of questions asked during in-class labs– Use labs as a way to encourage best practices
(frequent commits to version control, TDD)
ICER 2015
Replication Materials:http://people.engr.ncsu.edu/sesmith5/216-labs/csc216_labs.html
Replication Materials:http://people.engr.ncsu.edu/sesmith5/216-labs/csc216_labs.html
Thank You!
Questions?Comments?Concerns?
Suggestions?
ICER 2015