collecting and analyzing data: measuring student success rebecca orr, ph.d. professor of biology


Post on 14-Dec-2015




1 download

Embed Size (px)


  • Slide 1

COLLECTING AND ANALYZING DATA: MEASURING STUDENT SUCCESS Rebecca Orr, Ph.D. Professor of Biology Slide 2 Agenda Efforts made to improve student success: Online homework using MasteringBiology Peer tutoring Online quizzing Strategies for data collection: Preparing to begin Modifying as collection progresses Involving your students Case study: Impact of pre-exam quizzing on student outcomes Lessons learned Slide 3 Student Success Challenges Have you ever heard: I studied so hard, and it just didnt work! My question: How can student success be improved in the introductory biology (majors) course? Slide 4 My Evolution of Mastering Use Began using Mastering to create assignments and made them available for optional student use. Student feedback indicated a trend: Successful students were using Mastering Course evaluation surveys by students said: Make Mastering mandatory! Less successful students did not tend to utilize Mastering Moved to making Mastering mandatory by assigning a portion of lecture credit (10%) to Mastering. Slide 5 Is There a Significant, Positive Correlation Between Completing MasteringBiology and Exam Scores? Students required to complete MasteringBiology homework assignments (10% lecture grade). Overall exam average was compared to percentage MasteringBiology completed by end of semester. Spearman's Rank Order correlation was run: (rs(138) = 0.571, P =.000) A statistically significant, positive correlation between percentage of MasteringBiology homework completed and exam average was found. Slide 6 Results Students completing more work in MasteringBiology were more successful on exams. Slide 7 Problem: How to Increase Student Success in Introductory (Majors) Biology? My Strategy, Spring 2011 semester: Evaluate the efficacy of providing peer tutors vs. requiring MasteringBiology homework (using exam scores to measure) Offered optional, online pre-exam quizzes to all using MasteringBiology Results: Peer tutoring appears to have no benefit over offering required MasteringBiology homework. Significant, positive correlation between participating in optional quizzing and increased exam score observed. Slide 8 Cognitive Science of Learning Introduction of Desirable Difficulties: Conditions of instruction that appear to create difficulties for the learner, slowing the rate of apparent learning, often optimize long-term retention and transfer. Slide 9 Testing Serves as Learning Events Testing as a learning vehicle vs. an assessment vehicle. Evidence shows: Practice of recalling information is more powerful than re-presentation of information. Reality is: Students re-present information to themselves (i.e., reading a chapter over and over again, underlining things, etc.), focusing on storage rather than retrieval. Slide 10 Can An Online Homework Platform Be Used For Quizzing to Increase Student Success? Introduced required pre-exam quizzing delivered via MasteringBiology Students required to take pre-exam quizzes during the Fall 2011 semester (n=199) and Spring 2012 semester (n=174) My Question: Does taking pre-exam quizzes using Mastering result in significantly higher exam scores? Slide 11 Preparation Before beginning, set up spreadsheet to collect data necessary to answer questions. My predesigned columns included: Name (deleted prior to sending to statistician) Exam # and grade Number of pre-exam quizzes taken Grade on each pre-exam quiz Set up color coding to designate drops and not in my course Data gathered by exporting grade book from Mastering. Slide 12 Student Participation Compliance in Required Pre-exam Quizzing Fall 2011 & Spring 2012 % Students Completing Quizzes prior to Exam % Quizzes Completed Exam 1 n= 362 Exam 2 n= 339 Exam 3 n= 321 Exam 4 n= 305 100%78.559.962.959.3 67%16.322.4** 50%**26.525.9 33%3.011.5** 0%2.27.410.314.8 Slide 13 Comparison of Exam Averages 100% Quiz Takers to 0% Quiz Takers Fall 2011 Spring 2012 Slide 14 Comparison of Exam Averages by Percentage of Quiz Taking: Spring 2012 Slide 15 Analysis by Groups Scores on Exams 1 and 2 were averaged, and students were grouped by these averages. Groups Established: Pass: 70%+ Fail: