scientifically informed digital learning interventions
DESCRIPTION
Scientifically Informed Digital Learning Interventions. Financial and Intellectual Support: The William and Flora Hewlett Foundation The National Science Foundation A.W. Mellon Foundation Carnegie Mellon University. One Example: The Open Learning Initiative at Carnegie Mellon. The Challenge. - PowerPoint PPT PresentationTRANSCRIPT
Scientifically Informed Digital Learning Interventions
Financial and Intellectual Support:• The William and Flora Hewlett Foundation• The National Science Foundation• A.W. Mellon Foundation• Carnegie Mellon University
One Example: The Open Learning Initiative at Carnegie Mellon
The Challenge• To learn ways to design and build fully
web-based courses which by rigorous assessments are proven to be as good or better than traditional teaching methods
• Why?– Increasing access– Improving effectiveness– Providing flexibility for faculty and students– Containing costs
A Flaw and an Opportunity
• Current structure of higher education presents substantial roadblocks to the application of proven results and methodologies from the learning sciences
• eLearning interventions, developed by teams rather than individuals, are more conducive to making the practice of education more scientific and effective
OLI Guiding Assumptions• Digital learning interventions can make a
significant difference learning outcomes• Designs grounded in contemporary learning
theory and scientific evaluation have the best chance of achieving that goal
• A possible, acceptable outcome of the OLI efforts is failure or mixed failures and successes – we are doing “action research,” not promoting eLearning for its own sake
OLI Guiding Assumptions• Formative assessment will be a major
feature (and a major component of the cost) of the designs and iterative improvements of the courses
• IT staff working with faculty is too limited a partnership – learning scientists, HCI experts, and assessment experts must be part of design, development, production and iterative improvement
Open Learning Initiative Courses• Statistics• Modern Biology• Chemistry• French• Engineering Statics• Causal and Statistical Reasoning• Economics• Logic and Proofs• Physics• Empirical Research Methods• Computational Discrete Mathematics
Try it Yourself
• http://www.cmu.edu/oli
• Don’t expect an “OCW experience”…this project has a different set of goals than OCW
• “Clicking around” will be unsatisfying: these interventions are designed to support a novice learner in acquiring knowledge working on their own
Key Elements in OLI Courses
• Theory Based: Course and individual lesson designs based on current theories in the learning sciences
• Feedback Loops: Courses record student activity for robust feedback mechanisms
• Diversity of Perspectives, Roles and Contexts: Courses developed and deployed by teams that include faculty content experts, learning scientists, software engineers
Theory Based: Build on Prior/Informal Knowledge
Theory Based: Provide Immediate Feedback in the Problem Solving Context
Theory Based: Promote Authenticity, Flexibility & Applicability
• Learning environments with ambiguous problems that require flexible application of procedural knowledge
Feedback Loops in Learning
Evaluation• Chemistry: Post-test scores by treatment group show
significant positive correlation for the OLI treatment. Most significant indicator was time spent in Virtual Lab Activities – made all other variables drop out.
• Biology: End of the 3rd week showed an advantage for the OLI section. There was a positive and significant association between students’ time spent working on particular activities and performance on quiz questions testing the corresponding topics even after total time with OLI has been regressed out
Evaluation• Statistics 1st Study:
n Average % correct
Pre 488 43.3
Post 488 51.2
Increase: 7.9% Increase: 7.9% [t(487) = 13.8, p <.001][t(487) = 13.8, p <.001]
n Average % correct
Pre 24 55.8
Post 24 66.5
Increase: 11.7%Increase: 11.7%[t(23) = 4.7, p <.001][t(23) = 4.7, p <.001]
CAOS Sample:CAOS Sample:
CMU OLI Course Sample:CMU OLI Course Sample:
Evaluation
Measured learning Outcome
% correct CAOS % correct CMU
Pre Post Pre Post
Box plots provide accurate estimates of % data above & below only for quartiles
22.2 22.2 22.2 50.0
Correctly estimate and compare SD’s for different histograms.
31.5 46.4 66.7 83.3
41.8 46.4 59.3 75.0Correlation does not imply causation
51.9 49.4 48.1 70.8
Calculating appropriate conditional probabilities given table of data
49.6 47.4 70.4 70.8
Evaluation
Accelerated Learning Study
• Taught Carnegie Mellon Introductory Statistics course in a blended mode (one in class meeting per week) in half a semester
• The OLI Statistics course was the “textbook”• OLI course provided the professor immediate
feedback on students’ performance• We compared learning outcomes in the two
different treatments
Accelerated Learning Study• OLI students significantly outperformed
Traditional “control” students on the CAOS post-test. CAOS Post-test
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
OLI TRAD
Pro
port
ion
Corr
ect
Accelerated Learning Study• OLI students showed significantly greater gains (pre
to post) than the Traditional “control” students on the CAOS test.
0
0.05
0.1
0.15
0.2
0.25
OLI TRAD
Gain
Score
Student Satisfaction
– End of course survey for online section:
• All students reported at an increase in their interest in statistics.
• 75% Definitely Recommend
25% Probably Recommend
0% Probably not Recommend
0% Definitely not Recommend
Feedback Loop – Current Research
Instructors can use such data to adjust their teaching to students’ needs.
Learning Curve Analysis on Stoichiometry Data
The Vision – Digital Dashboard for Teaching and Learning:• Instructor assigns students to work
through online instruction• System collects data as students work • System automatically analyzes and
organizes the data to present instructor with the students’ current “learning state”
• Instructor reviews this data summary and adapts instruction accordingly
The Anticipated Benefits
• Instructors get a window onto students’ progress
• They can adapt their teaching accordingly
• Students get better feedback to monitor and adjust their learning
• Strengthens the student-instructor connection
Core OLI Community• Faculty Content Experts• Learning Scientists• Human Computer Interaction• Software Engineers• Evaluation/Assessment Specialists• Learners• A community of scholars from diverse
disciplines who are committed to improving quality and access to instruction. The collaborative nature of the OLI course design process inspired participating faculty to rethink their approach to classroom teaching.
“Improvement in post-secondary education will require converting teaching from a ‘solo sport’ to a community-based research activity”
Herbert Simon
www.cmu.edu/oli
[email protected]@cmu.edu
(Candace Thille – Director)