the longitudinal student assessment project (lsap)
TRANSCRIPT
The Longitudinal Student Assessment Project (LSAP)
2
Mary Pat Wohlford-Wessels, Ph.D. Assistant Dean Academic Quality & Medical Education Research College of Osteopathic [email protected]
Diane Hills, Ph.D. Associate Dean for Academic Affairs
College of Osteopathic [email protected]
3
Introduction
This presentation provides information about the development of a database of select student variables that can be used in medical education research.
The variables included in the database to date represent an initial attempt to begin to answer questions related to admission decisions and academic success.
Data regarding the performance of students as they progress through the curriculum are included as well.
4
A systematic review of the literature by Ferguson et al (2000), revealed:
That though previous academic performance is a good predictor, it is not perfect.
Their systematic review indicates that a strategic learning style, white ethnicity, and gender (female) were all associated with success.
5
Research Questions
Is there a relationship between undergraduate Science GPA and success in Year 1, Year 2, and on COMLEX 1 and 2 ?
Is there a relationship between MCAT score and success in Year 1, Year 2, and on COMLEX 1 and 2?
Can the Product used by the admissions committee (Undergrad science GPA X MCAT) be used to predict success?
Can performance in Year 1 and or Year 2 be used to predict success on COMLEX 1 or COMLEX 2?
How do male and female students differ in terms of preparation for medical school (science GPA and MCAT) and performance during years 1 & 2 and on COMLEX 1 and 2?
Is there a difference in admission metrics for students who rank in the top 25 % of the class and those who rank in the bottom 25% of the class.
The initial dataset was developed for the Class of 2005, 2006, and 2007. A total of 590 student files are included in the initial database. The database was developed to help answer the following questions:
6
Initial Data Set Student name Total MCAT score Undergraduate science GPA Admission Product (total MCAT x Undergraduate Science GPA) Rank first year GPA first year Rank second year GPA second year Gender COMLEX 1 mean score COMLEX 2 mean score Class
7
Additional data added the second year
Cumulative undergraduate GPA Non-science undergraduate GPA Undergraduate Institution Code Undergraduate degree Undergraduate Major MCAT score components Ethnicity Age Marital status Basic science scores Physical Diagnosis scores COMLEX 2 PE scores COMLEX 3 scores
8
Results
Gender: 45 % of the students included in the database are female.
Weak correlations existed between: Undergraduate science GPA and 1st year GPA
Undergraduate science GPA and 2nd year GPA Undergraduate science GPA and COMLEX 1 Undergraduate science GPA and COMLEX 2
9
Results
Moderate correlations existed between: 1st and 2nd year GPA
1st year GPA and COMLEX 1 2nd year GPA and COMLEX 2 COMLEX 1 & COMLEX 2.
Though the undergraduate science GPA and the MCAT score are both used in the admission product, the undergraduate GPA has slightly higher predictive value.
10
Results
Significance testing indicates that there is a difference between female and male students on total MCAT scores, the admission product, and on COMLEX 1 scores.
On all three metrics, male students outperformed female students.
11
Results
Significance testing reveals that those students who are ranked within the top quartile of the class as opposed to those who rank in the bottom quartile of the class have significantly higher undergraduate science GPA’s, admission product scores, 1st year GPA’s, COMLEX 1 scores, 2nd year GPA’s, and COMLEX 2 scores.
It is important to note that those in the top quartile do
not have significantly higher MCAT scores.
12
Conclusion
It is clear that DMU-COM students are all competitive, but this project has provided some clarity related to the predictive value of select metrics, statistical differences between performance quartiles, and identified differences between female and male students.
Over the next academic year, with the inclusion of select additional variables (verbal MCAT, total undergraduate GPA, ethnicity, age, etc.) additional questions can be answered.