erin baumgartner & ava howard biology department western oregon university
DESCRIPTION
Data Driven Decision-Making. Erin Baumgartner & Ava Howard Biology Department Western Oregon University. How do you get a group of passionate faculty with diverse backgrounds to agree on instructional practice?. DATA!. For Example…. What’s better for students- many small or few large exams? - PowerPoint PPT PresentationTRANSCRIPT
Erin Baumgartner & Ava Howard Biology Department
Western Oregon University
Data Driven Decision-Making
How do you get a group of passionate faculty with diverse backgrounds to agree on instructional practice?
DATA!
For Example…• What’s better for students- many small or few large exams?
• What’s better for students to have the same lecture and lab instructor or different instructors?
• What’s better for students – multiple choice or open-ended assessments?
• How do we improve the laboratory experience?
• How consistent are exam practices between instructors? Does it matter?
• Is there a difference between 4-item or 5-item response choices on multiple choice questions?
• How are we serving non-traditional, underrepresented, and other student groups?
Hypothesis:
• More frequent, smaller exams will improve student study habits and performance and decrease anxiety.
Exam Size & Frequency
• Control-comparison– Two midterms, 90 points, 50 minutes– Four miniexams, 45 points, 25 minutes
• Quantitative: – Embedded indicator questions on exams– Student survey – Likert response scale
• Qualitative: – Student comments
Methods:
• Decrease anxiety – No • Improved study habits – No• Improved performance – Minimal
… But students love them
Results:
ConclusionsConclusions:
• Students with same lecture and lab instructor gain more content knowledge and improve attitudes about science.
Lecture and Lab Instructors
Hypothesis:
• Naturally occurring comparison groups– Students with same lecture-lab– Students with different lecture-lab
• Quantitative: – Proportion of correct answers on concept inventory– Student attitude survey – Likert response scale– Two-sample t-test overall comparison
• MANOVA across demographic groups
Methods:
same lecture-lab different lecture-lab0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Average Post-Course Content Scorep = 0.7481; t = 0.3215 w/265 df
Condition
Aver
age
perc
ent s
core
same lecture-lab different lecture-lab0
1
2
3
4
5
Average Post-Course Attitude Scorep = 0.3453; t = 0.9456 w/237 df
Condition
Aver
age
Liker
t Sco
re
Results:
ConclusionsConclusions:
• Students perform better on open-ended assessment items.
Open Ended:What is a sphincter?Identify a place in the body where you might find one.
Multiple Choice: What is the role of the sphincters in the digestive system?
Hypothesis:
Types of Exam Questions
A. Secrete enzymesB. Prevent food from moving too quickly
or moving backward through the system
C. Keep food moving through the systemD. Adjust pH in different parts of the
system
• Within group comparison – Exams consisted of multiple choice and open-
ended items (short-answer, fill-in-the-blank, labeling, essay)
• Quantitative: – Correlation of performance on item type against
overall performance
Methods:
20 30 40 50 60 70 80 90 1000
10
20
30
40
50
60
70
80
90
100
Exam item type score correlated with total exam score
Percent MC Score
Linear (Percent MC Score)
Percent written score
Linear (Percent written score)
Overall exam score
Scor
e on
exa
m it
em ty
pe
Pearson correlation coefficient = 0.855
Pearson correlation coefficient = 0.900
Results:
Question Type Points each # on exam total points % allocation notes
multiple choice (MC)
2 16 32 53% 2 points per item
Matching 1 12 12 20% 1 point per item, combined into appropriate number of questions
Fill in the blank or label diagram
2 6 12 20% can be combined into fewer questions; or split- single word answers can be worth 1 point, while short (phrase or one sentence) answers worth 2
Open Ended (Short answer/ Drawing/ Essay)
4 1 4 7% can be divided into multiple parts or combined into one question with several elements
TOTALS: 35 60 100%
Conclusions:
5 items?1. A carefully formulated scientific
explanation that is based on a large accumulation of observations and has been tested and supported by multiple scientistsa. Controlb. Theoryc. Hypothesisd. Facte. Postulate
4 items? 1. A carefully formulated scientific
explanation that is based on a large accumulation of observations and has been tested and supported by multiple scientistsa. Controlb. Theoryc. Hypothesisd. Fact
• Morrison, Susan and Free, Kathleen. (2001). Writing multiple-choice test items that promote and measure critical thinking. Journal of Nursing Education 40, 17-24.
• Rodriguez, M.C. (2005). Three Options are Optimal for Multiple-Choice Items: A Meta-Analysis of 80 Years of Research. Educational Measurement: Issues and Practice, 24(2), 3-13.
Dissemination is Important
IRBs are Important
A Consistent Framework is Important
• Lindsay Biga• Karen Bledsoe• Jim Dawson• Bryan Dutton• Emma Dutton• Amy Harwell• Kelly Kissane• Kristin Latham• Liz Martin• Jeffrey Snyder
Julie Grammer
Thanks!