data analytics slides - university of north carolina at
TRANSCRIPT
Using Student Data to Advance the University's
Teaching Mission
10:05- 10:55
Comments/Questions: pollev.com/carolina
KEVIN GUSKIEWICZDean of the College of Arts and Sciences
KELLY HOGANAssociate Dean of Instructional InnovationTeaching Associate Professor of Biology
Using data to track student outcomes: a case study in transitioning to high structure active learning
STEM classes
4 DEPARTMENTS (BIOLOGY, CHEMISTRY, PHYSICS and MATH)
12 GATEWAY COURSES TRANSFORMED
25FACULTY APPRENTICES IN SIX SEMESTERS
>6,000 STUDENTS PER YEAR
Improved Learning…
45FACULTY IN LEARNING COMMUNITIES
New tools needed
PRACTICE AND FEEDBACKLISTENING TO CONTENT
Use a new innovation
“WHAT BARRIERS EXIST?”
TIME TO DEVELOP THE COURSE.
KNOWLEDGE ABOUT PEDAGOGY
SUPPORT FROM A GROUP OF COLLEAGUES
Learn about innovation
Convinced of value of innovation
Decide to try the innovation
Stages of Innovation Process:
1. MENTOR-APPRENTICE RELATIONSHIPS
2. FACULTY LEARNING COMMUNITIES
TIME (COURSE RELEASE)KNOWLEDGE/PRACTICE WITH PEDAGOGY
SUPPORT FROM COLLEAGUES DURING CULTURAL SHIFT
INTERVENTION:
Before class
During class
After class
Readings, videos, online homework, online discussions
problem solving, individual work, peer instruction, polling/clickers, designing experiments, etc
problem sets, online quizzes
DESIRED OUTCOME FOR PEDAGOGY:High Structure Active Learning
Organic Chemistry I. Nearly identical final exams given.
CHEMISTRY: EVIDENCE THAT INCREASING STRUCTURE IMPROVES PERFORMANCE FOR ALL
M.T. Crimmins and B. Mdkiff J. Chem. Educ., 2017, 94 (4), pp 429–438
Additionally, D/F rates dropped from 18% to 9.5% for all students
PHYSICS: Increasing structure improves learning of specific concepts
Traditional structure
Traditional structure +
Some Active Learning
Lecture/Studio with Life Science Focus
D.P. Smith et al. American Journal of Physics 86, 862 (2018)
Preliminary findings across project:
13% increase in normalized learning gains with high structure, active learning compared to instructor centered design
(data from over 14,000 student measurements)
PHOTO CREDIT: KRISTEN CHAVEZ
Have the pedagogical/curricular changes impacted different student groups?
PHOTO CREDIT: KRISTEN CHAVEZ
Biology: Some student groups disproportionately benefit from increased structure
3.2 - 3.7 % increase
6.1 % increase
6.3 % increase
All s
tude
nts
Firs
t gen
erat
ion
stud
ents
Blac
k st
uden
ts
Eddy, S. L.; Hogan, K. A. CBE Life Sci. Ed. 2014, 13, 453-468
Failure rates dropped by 40%.
Those results required obtaining data from
institutional data requests…
What if we all had a way to see data for our own classes routinely?
Change in Grade Distribution for First Generation, Pell-Eligible Students in
One Redesigned Course (Biol 202)
0102030405060
Fall 2012 Fall 2016
A B C D F W77% successful
82 % successful
n=22 n=27
perc
ent o
f stu
dent
s
Data retrieved from MCAD
My Course Analytics Data (MCAD)
Data are available from Fall 2010 to the most recent semester during which the course was taught and completed.
Example data from the dashboard show the percent of students in different demographic categories. Hovering over the bars will give the percentages and total number in each subgroup.
Learn more: https://cfe.unc.edu/mcad/
My Course Analytics Data (MCAD)Example data. Dashboard users customize their queries to examine individual courses in specific semesters or to combine multiple semesters.
A
A
B
B C
C D
D
Pell recipient
Non-Pell recipient
PEDAGOGICAL PRINCIPLE
Students come to the classroom with preconceptions about how the world works.
If their initial understanding is not engaged, they may fail to grasp the new concepts and information that are taught.
National Research Council: How People Learn. Washington, DC: National Academy Press, 2000, pp 14-15
MCAD’S ROLE
• In large classes, difficult to identify those individual preconceptions.
• One partial solution: • Look to the success of various groups in previous years, and
• Adjust the class content to reach those with less success in those years.
EXAMPLE: WOMEN IN ECONOMICS
• A national trend: fewer women majoring in Economics
• 2013: look “by hand” at grades I gave in Econ 101. Women seemed to have somewhat lower grades on average.
• Response:• Changing examples and applications – more music/design/advertising.
• Identifying/highlighting successful women in economics.
• Greater emphasis on engaging women in the classroom.
• By 2015: closer to parity.
• Today: with MCAD, such queries are done in seconds.
PROBLEMS
• Loose link between pedagogical goal and available categories.
• Only available for completed courses. “Formative assessment” would be very helpful.
FROM THE CHAIR’S PERSPECTIVE
• NB: Chairs don’t have access to other faculty’s data at present.
• In courses where we have challenges – differing success rates –we can quantify these differences.
• Over time, we can experiment with teaching techniques as a department to bring success rates in line with our high expectations for our students.
• Example: Eddy, Sarah L., and Kelly A. Hogan. 2014. “Getting Under the Hood: How and for Whom Does Increasing Course Structure Work?” CBE-Life Sciences Education 13 (3): 453–68.
Discussion “rules”.
Raise your hand. (Please limit your comments to 60 seconds or less, so that many people have an opportunity to join the conversation.)Write on a notecard and signal to the moderator
for pick up.Use pollev.com/carolina for anonymous
questions/comments.
We invite you to join into the conversation in multiple ways: