the power of quizzing
Post on 11-Feb-2016
36 Views
Preview:
DESCRIPTION
TRANSCRIPT
THE POWER OF QUIZZINGTHE POWER OF QUIZZING
Mark A. McDaniel Washington University
NCAT Meeting, March 24, 2009
Kaplan Test Prep and Admissions Survey
Eat Alaskan salmon for breakfast
Watch back-to-back episodes of ER
Wear Superman underwear
• Laboratory research: Taking a test can serve as a learning opportunity, and enhance memory retention even to a greater extent than additional studying… the “testing effect”
• Central implication for course redesigns:
In college classes, low or no stakes quizzes (tests) can be used to enhance student learning (quizzes not just a tool to reveal student mastery)
Wildman, McDaniel & Anderson (2009)
In WEB-based Brain and Behavior university course, each week students covered one chapter (40 pages)
14 central facts from chapter presented to students in one of three ways (as an assignment):
Read the facts to review Multiple choice quiz on the facts Short answer (fill in blank) quiz on the facts
14 other central facts not presented
Summative Assessments (course exams): 84 Item Multiple Choice Exams over Three Chapters
Experiment 1
1. Summative tests manipulated match between quiz stem and test stem: Quiz Stem: Touch receptors (for nocioception and hapsis) on the skin have
their cell bodies in ____________ and connect in the spinal cord to ascending spinothalamic cells.
Exam Stem: Same Touch receptors (for nocioception and hapsis) on the skin have
their cell bodies in ____________ and connect in the spinal cord to ascending spinothalamic cells.
A. dorsal root ganglia, B…., C…, D…..
Different Touch receptors (for nocioception and hapsis) on the skin have their cell bodies in the dorsal root ganglia and connect in the spinal cord to ___________.
A……, B……,C……, D. ascending spinothalamic cells
2. Quizzes presented on web Ungraded Correct answer feedback provided
3. Repeated quizzing: students required to take quizzes four times. (Maximum credit for quiz taking was earned only by taking quizzes four times)
Unit Test Performance
SA = Short answer quiz, MC = Multiple choice quiz, QR = reading, NX = no-exposure
Experiment 1 Different Stems
n = 16
0.76 0.76 0.71 0.68
00.10.20.30.40.50.60.70.80.9
1
SA MC QR NX
SA = Short answer quiz, MC = Multiple choice quiz, QR = reading, NX = no-exposure
Experiment 1 Same Stems
n = 16
0.87 0.850.71 0.68
00.10.20.30.40.50.60.70.80.9
1
SA MC QR NX
Experiment 2Can These Robust Test-Enhanced Learning Effects Be
Replicated and Extended to condensed course format?
• Condensed summer-school course (7 weeks) Individual students could start late and further condense the course
(essentially complete course in 3-4 weeks)
• Similar to Experiment 1Summative tests manipulated match between quiz stem and test stem:
Different or Same
Students required to take quizzes four times (Maximum credit for quiz taking was earned only by taking quizzes four times)
Unit Test Performance
SA = Short answer quiz, MC = Multiple choice quiz, QR = reading, NX = no-exposure
Experiment 2 Different Stems
n = 27
0.75 0.75 0.76 0.65
0
0.2
0.4
0.6
0.8
1
SA MC QR NX
SA = Short answer quiz, MC = Multiple choice quiz, QR = reading, NX = no-exposure
Experiment 2 Same Stems
n= 27
0.90 0.920.76
0.65
00.10.20.30.40.50.60.70.80.9
1
SA MC QR NX
Summary
• Repeated quizzing produces robust learning effects on classroom summative assessments
• Multiple choice quizzes are as effective as short-answer quizzes, when summative assessment is multiple choice and quizzes repeated
• With same stems on quizzes and final test, quizzing produces better performance than presenting target content for additional study
• Can quizzing enhance richer learning of conceptual information (application of content)?
• Will application quiz items promote learning of related factual content?
Short-answer quizzes (web-based course)FactualWhat is the most complex human behavior?
AppliedIf an otherwise normal child does not start talking until after his peers, what do we expect about how long it might take this child to learn to speak fluently?
Performance on Exam Multiple Choice Questions as a Function of Quiz Type
0.84
0.650.74
0.64
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Applied Factual None
Quiz Type Question
Prop
ortio
n Co
rrec
t at E
xam
Exam Question Applied
Exam Question Factual
Performance on Exam Multiple Choice Questions as a Function of Quiz Type
0.840.72
0.650.74
0.810.64
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Applied Factual None
Quiz Type Question
Prop
ortio
n C
orre
ct a
t Exa
m
Exam Question AppliedExam Question Factual
Summary
• Quizzing (short-answer) can promote transfer—it enhances performance on related questions
Does Test-Enhanced Learning Extend to Typical Lecture Courses
and Very Able Learners?• Washington University Introductory Psych
Class (summer)• Intervening quizzing: short answer quizzed not quizzed• Exam question types: Multiple Choice, Short
Answer
McDaniel and Sun(2008)
12 Short answer items
(12 Non-quiz items)
QUIZ SENT BYE-Mail (2/week) UNIT EXAM
Mult. choice (16): 8 same wording and 8 different wording from quiz
Short answer (8): 4 same wording and 4 different wording from quiz
End of
Week
Class lecture;
reading assignment
N=21
Daily
Proportion correct on exams as function of quizzing, exam question type, and match of quiz
and exam question stems
00.10.20.30.40.50.60.70.80.9
1
Same Different Same Different
MC SA
Prop
ortio
n C
orre
ctly
R
ecal
led
Quizzed
Not Quizzed
IS TEST-ENHANCED LEARNINGPERSISTENT OVER LONG DELAYS?
Columbia Middle School project
• 6th, 7th, and 8th grade science classes
• Some facts on each lesson are quizzed 3 times; comparable facts are not quizzed
• Consider recall on tests at ends of chapters, end of the semester and end of the year
0.91
0.76
0.68
0.74
0.69
0.61
0.5
0.6
0.7
0.8
0.9
1
Chapter Exams End of the Semester End of the School Year
Pro
porti
on R
ecal
led
TestedNon-Tested
COLUMBIA MIDDLE SCHOOL PROJECT8TH GRADE SCIENCE
• Can learner initiated quizzes (retrieval) enhance learning (when course constraints preclude any quizzing)?
Learner controlled testing
• Read (study)• Recite (recall material to oneself)• Review (reread the material)
McDaniel, Howard, & Einstein
(2009, Exp. 2)
• Three study conditions:– Read, recite, review– Note-taking– Read-twice control
• Each participant read two complex, ~900 word articles– Brakes & Pumps, as in Mayer & Gallini (1990)
• Students were tested on one of the articles immediately
• After a one-week delay, students were tested on both articles
• Testing consisted of:– Free recall– Multiple choice (4 alternative)– Short-answer inference questions
**
**
**
**
*
* **
Summary• Participants who used the 3R strategy
performed significantly better on a range of classroom type tests relative to reading twice or note-taking (excluding immediate inference performance)
• 3R more efficient (takes less time) than note-taking for enhancing problem solving/inference
Summary of major findings
• Basic test-enhanced learning demonstrated in college courses and classrooms
• Found even when learning at high levels.
• Similar learning benefits were not always found from additional exposure per se
• Self-testing is effective for learning
Many reasons to expect tests to confer learning benefits in educational contexts
Retrieval itself produces retrieval practiceTest itself requires student to engage in the
material (Sikorsky et al., 2001)Signals information to target for study.
(improves metamemory—both accuracy of what you know and control of study)
Leads to more (and spaced) studying—keep up with the material better (Johnson & Kiviniemi, 2009)
Acknowledgements
FundingInstitute of Education SciencesGrant #R305H030339
James S McDonnell Collaborative Activity Grant
top related