1. A novel web-based learning platform for accelerated knowledge retention in higher education Jeremiah R. Cohen, B.S. David Geffen School of Medicine at UCLA Massachusetts Institute of Technology, 2009 Memorang.io George J. Courtsunis, B.S. Massachusetts Institute of Technology, 2009 Memorang.io June 5, 2014

2. The Problem Increasing volumes of knowledge to retain Memorization tools have not fundamentally changed in 10 years Most of these tools center around flashcards Flashcards represent knowledge in a one-to-one relation, while integrated knowledge requires many-to-many relations. Lack of detailed feedback for metacognition Many students lack structure Create Evaluate Analyze Apply Understand Rote memory 3. Jeremiah George (formerly Enki Learning) 4. Why not automate Gagnes 9 Levels of Learning? Gain attention (Reception) Informing learning of the objective (Expectancy) Simulating recall of prior learning (Retrieval) Presenting the stimulus (Selective perception) Providing learning guidance (Semantic encoding) Eliciting Performance (Responding) Providing Feedback (Reinforcement) Assessing Performance (Retrieval) Enhancing Retention and Transfer (Generalization) 5. Strategy 6. Content 7. Retrospective Cohort Inclusion criteria Studying for USMLE Step 1 Board Exam 100+ questions answered 2+ study sessions Last 4 weeks Data items recorded Answers, mode of study, accuracy, study sessions, mastery score, specific content studied 8. Overview 558 students identified 552,631 facts studied 0 500 1000 1500 2000 2500 3000 3500 25th 50th 75th FactsStudiedPerStudent Percentile 9. Facts Studied per Learning Mode 1. Multiple Choice 2. Flashcards ~ Matching 3. Candy Cannon 79% 10% 10% 1% Multiple Choice Flashcards Matching Candy Cannon 10. Accuracy 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 25th 50th 75th Accuracy Percentile Overall Multiple Choice Flashcards Matching Candy Cannon 11. Weighted Accuracy 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 25th 50th 75th Accuracy Percentile Overall Multiple Choice Flashcards Matching Candy Cannon 12. Accuracy per Subject Mastery R = 0.9928 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 20% 40% 60% 80% 100% Accuracy Mastery 13. Accuracy per Subject Mastery per Mode 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 20% 40% 60% 80% 100% Accuracy Mastery Multiple Choice Flashcards Matching Candy Cannon 14. Case Study: Cellular Proteins Inclusion criteria Set with a large amount of study activity Reached exactly 70% mastery Data items recorded Answers, mode of study, accuracy, study sessions, mastery score, specific content studied Selection: 17 students 196 facts per student to reach 70% 15. Case Study: Facts Studied per Learning Mode 61% 24% 14% 1% Multiple Choice Flashcards Matching Candy Cannon 16. Case Study: Accuracy per Subject Mastery 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% 20% 40% 60% 70% Accuracy Mastery 17. Case Study: Facts studied to reach 70% 80% 82% 84% 86% 88% 90% 92% 94% 96% 98% 100% 0 50 100 150 200 Accuracy Number Facts Studied to reach 70% Mastery 18. Conclusions When presented with multiple learning modes, medical students show an overwhelming preference for multiple choice questions over other modes Mastery scores can be used to predict accuracy At lower levels of subject mastery, matching is the easiest At moderate-to-high levels of mastery, all modes are equal Flashcards are not more effective than any other learning mode, and students may benefit from having more flexibility. 19. Next steps Correlate study data with exam scores Real-time data tracking for students and educators Leverage data to offer personalized recommendations Determine optimum answer events to thoroughly master a concept Experiment with different strategies and modalities (e.g. spaced repetition vs. original order vs. randomized shuffle). Compare mobile device vs. desktop/laptop studying 20. Thank you! We welcome your feedback and questions yermie@memorang.io george@memorang.io