powers gre 98-09r ets-rr-01-10 final

GRE®

R E S E A R C H

May 2001

GRE Board Research Report No. 98-09R

ETS Research Report 01-10

Validity of GRE® General Test Scoresfor Admission to Colleges of

Veterinary Medicine

Donald E. Powers

Princeton, NJ 08541

Validity of GRE® General Test Scores for Admission to Colleges of Veterinary Medicine

Donald E. Powers

GRE Board Report No. 98-09R

May 2001

This report presents the findings of aresearch project funded by and carried

out under the auspices of theGraduate Record Examinations Board.

Educational Testing Service, Princeton, NJ 08541

********************

Researchers are encouraged to express freely their professionaljudgment. Therefore, points of view or opinions stated in Graduate

Record Examinations Board reports do not necessarily represent officialGraduate Record Examinations Board position or policy.

********************

The Graduate Record Examinations Board and Educational Testing Service arededicated to the principle of equal opportunity, and their programs,

services, and employment policies are guided by that principle.

EDUCATIONAL TESTING SERVICE, ETS, the ETS logos,GRADUATE RECORD EXAMINATIONS, and GRE are

registered trademarks of Educational Testing Service.

Educational Testing ServicePrinceton, NJ 08541

Copyright 2001 by Educational Testing Service. All rights reserved.

Abstract

This paper documents a case study of the validity of the Graduate Record Examinations

(GRE®) General Test in a comprehensive sample of veterinary medical colleges. Extensive and

complete data available from these schools allowed us to control several artifacts that are typical

of �one-shot� validation studies and thus enabled a relatively definitive assessment of the test�s

validity. Overall undergraduate grade point average (GPA), undergraduate GPA in the last 45

hours of courses, and GRE verbal, quantitative, and analytical scores were examined individually

and in combination for their ability to predict success in veterinary school. For each of 16

veterinary medical colleges, statistical methods were applied to correct for the effects of range

restriction in the predictors and unreliability of the criterion, and the results were summarized

across all schools. When corrections were made for both range restriction and unreliability, the

resulting validity coefficients (median multiple correlations) were .53 for three GRE scores,

when used together; .56 for overall undergraduate GPA; and .72 for GRE scores and overall

undergraduate GPA considered together. Adding GRE scores to undergraduate GPA increased

the amount of variance explained, on average, by about 18% � a proportion that can be regarded

as being �medium� to �large.�

Keywords: Graduate Record Examinations, GRE, veterinary medical colleges, test validity,predicting success, graduate admissions

Acknowledgements

Special thanks go to each of the following people, without whose help or advice this

report would not have been possible:

• Kurt Geisinger for suggesting that veterinary medical colleges would provide a goodopportunity for GRE validity studies

• Beth Winter for helping us gain entry to veterinary schools, for facilitatingcommunication with them, and for advice throughout the course of the study

• Mary Herron and Curt J. Mann, of the Association of American Veterinary MedicalColleges, for their support and advice

• Julie Lind and Don Elder for information about the Veterinary Medical CollegeApplication Service and for facilitating the transfer of data needed for the study

• Tony Confer for helping us locate prior validity studies of veterinary medical collegeadmissions

• Sheila Allen, Linda Blythe, David Bristol, Barbara Coats, Ronnie Elmore, WilliamFenner, Robert Hansen, Kathyrn Kuehl, Michael Lorenz, Sherry McConnell, ClaireMiceli, Kenneth Myers, Charles Newton, Denise Ottinger, Gerald Pijanowski, JohnRhoades, Rebecca Russo, James Thompson, Eldon Uhlenhopp, John Van Vleet,Yasmin Williams, and Yvonne Wilson for acting as our contacts and for facilitatingdata collection and other aspects of the study at their institutions

• Susan Leung for coordinating data collection

• Michelle Najarian for managing the database and conducting the analyses; Tom Jirelefor advice on analyses; and Fred Cline for producing graphics describing analyses

• Charlie Lewis for advising us on the data analysis

• Ruth Yoder for helping to process data, produce data collection instruments, andcoordinate numerous other aspects of the project

• Leona Aiken, Narayan Bhat, Nancy Burton, Brent Bridgeman, Carol Dwyer, andDiane Halpern for providing helpful feedback on an earlier draft of this report

• The Graduate Record Examinations (GRE®) Board and the GRE Research Committeefor their support of this effort

Executive Summary

Traditionally, it has been difficult to conduct good validity studies for graduate

admissions. Graduate-level validity studies have been hampered in several ways � for example,

by the availability of relatively small samples, by the use of unreliable criteria, and by the

influence of selection procedures that restrict the range of test scores and other predictors of

success. These factors can serve either to depress validity estimates or to make them appear more

variable across graduate institutions than they really are.

This study took advantage of a unique opportunity to address several shortcomings

typical of many previous Graduate Record Examinations (GRE®) validity studies. This

opportunity arose in the context of admissions to colleges of veterinary medicine, most of which

require applicants to submit scores from the GRE General Test. From the standpoint of

conducting validity studies, several features of veterinary schools made such an opportunity

attractive: (a) larger class sizes than are typical of most graduate departments; (b) relatively

uniform curricula across institutions; (c) the availability of information about applicants; and (d)

an opportunity to assess the reliability of criteria. These features enabled us to statistically

correct for several of the factors that can lead to inaccurately low estimates of true validity. They

also allowed us to estimate the degree to which apparent between-school variation in validity

estimates is due to statistical artifacts, rather than to real differences in the predictability of

success from school to school.

As anticipated, the highly selective nature of veterinary medical school admissions was

clearly apparent from the data. There was ample evidence of factors � for example, restriction in

the range of test scores and undergraduate grade point averages (GPAs) � that can result in

serious underestimates of the validity of preadmission measures. Even with the dampening

effects of these factors, however, both undergraduate GPA and GRE General Test scores

exhibited significant relationships with first-year grades in veterinary medical school. Moreover,

when used together, grades and test scores constituted a more powerful predictor combination

than did either one used alone. Adding GRE scores to grades increased prediction by an amount

that social scientists usually consider as being �medium� to �large.�

More importantly, when statistical corrections were made to counteract the attenuating

effects of selection and criterion unreliability, validity estimates increased significantly. When

fully corrected for both range restriction in the predictors and unreliability in the criterion, the

combination of undergraduate GPA and GRE General Test scores accounted, on average, for

more than half the variation in first-year veterinary school grade averages.

By focusing on a particular context � veterinary medical school admissions � we were

fortunate to obtain the cooperation of a majority of U.S. veterinary colleges of medicine, and

hence information about a preponderance of first-year veterinary medical students in the 1998-99

academic year. Thus, unlike the results of validity studies based on single institutions, the

findings reported here are reasonably representative of a specific universe of interest � in this

case, veterinary medical schools. Moreover, an analysis of the variation among schools

suggested little reason to question the generalizability of our findings across veterinary medical

schools: With the exception of one school, validity estimates seemed to apply about equally well

to all of the schools that participated in the study.

Table of Contents

PageIntroduction ..................................................................................................................................... 1

Factors Affecting the Estimation of Validity Coefficients .......................................................... 2Validity Generalization................................................................................................................ 4Veterinary Medical School Admissions ...................................................................................... 5Relevant Research........................................................................................................................ 6Research Questions...................................................................................................................... 6

Method ............................................................................................................................................ 7Sample Selection.......................................................................................................................... 7Instruments/Data .......................................................................................................................... 9Data Analysis ............................................................................................................................... 9

Results ........................................................................................................................................... 12

Discussion ..................................................................................................................................... 32Limitations ................................................................................................................................. 34

Conclusion..................................................................................................................................... 35

References ..................................................................................................................................... 37

Appendix ....................................................................................................................................... 41

List of Tables

PageTable 1. Participating Universities .............................................................................................. 8Table 2. Correlations of Preadmissions Variables with First-Year Average in Veterinary

Colleges of Medicine .................................................................................................. 15Table 3. Means and Standard Deviations of Preadmissions Variables for Applicants and

Enrolled Students ........................................................................................................ 16Table 4. Correlations of Preadmissions Variables With First-Year Average in Veterinary

Colleges of Medicine .................................................................................................. 20Table 5. Correlations of Preadmissions Variables With First-Year Average in Veterinary

Colleges of Medicine .................................................................................................. 21Table 6. Correlations Among Preadmissions Variables for Applicants and Enrolled

Students ....................................................................................................................... 25Table 7. Student Perceptions of the First Year of Veterinary Medical School ......................... 28Table 8. Statistics for Individual Course Grades....................................................................... 31

List of Figures

Page

Figure 1. Distributions of validity coefficients before and after corrections (UGPAT as theundergraduate grade indicator).................................................................................... 17

Figure 2. Distributions of validity coefficients before and after corrections (UGPA45 as theundergraduate grade indicator).................................................................................... 18

Introduction

The Graduate Record Examinations (GRE®) General Test is a standardized test of verbal,

quantitative, and analytical reasoning that is designed, primarily, to facilitate admissions to U.S.

graduate schools. Since its inception in 1949, the original measure and several revisions have

been widely used and frequently studied (see, for example, Briel, O�Neill, & Scheuneman,

1993). In fact, according to Hambleton (1998), there have been some 1,500 studies of the

validity of the GRE General Test � plus one! The �one� (Sternberg & Williams, 1997) � a case

study of psychology graduate students at a prestigious academic institution (Yale University)

published in an esteemed scholarly journal (American Psychologist) by two prominent

psychologists � attracted considerable, though perhaps undue, attention. While understandable,

the degree of interest in this single study � and its largely negative results � flies in the face of

current conceptions of test validation, which is now generally viewed as the process of

accumulating evidence regarding the meaning and value of test score inferences (Messick,

1989). In short, one study does not a test validation make.

In addition to the attention it received, the Sternberg and Williams� (1997) study is also

notable with regard to the criticism it drew, much of which concerned professional requirements

for defensible validity studies. The ensuing, mostly critical commentary (Andre & Hegland,

1998; Cornell, 1998; Darlington, 1998; Kuncel, Campbell, & Ones, 1998; Melchert, 1998;

Miller, Barrett, & Doverspike, 1998; Roznowski, 1998; Ruscio, 1998; Thayer & Kalat, 1998)

suggested that, like numerous other validity efforts, the study was multiply flawed because it:

• relied on small samples

• employed unreliable criteria

• overgeneralized from a single, atypical department

• discounted the effects of compensatory selection

• failed to account for range restriction in the criteria and in the predictors

With these criticisms in mind, we undertook a case study of the validity of the GRE

General Test in a comprehensive sample of veterinary medical colleges. The extensive and

2

complete data available from these schools allowed us to control several of the artifacts that

affect many one-shot validation studies, and thus enabled a relatively definitive assessment of the

test�s validity.

Factors Affecting the Estimation of Validity Coefficients

While some of the factors mentioned above (e.g., the use of small samples) can give rise

to inconsistent results, others (e.g., range restriction, criterion unreliability, and compensatory

selection) can occasion serious underestimates of the validity of admissions measures. The

potential influence of each of the latter factors is discussed briefly below.

Range restriction. It has long been known (since Pearson, 1903) that sampling from a

population can, by curtailing the range of a variable, artificially depress its correlation with other

variables. Such restriction typically occurs in academic admissions when students are selected

from a larger pool of applicants � for example, on the basis of test scores. Generally, the effect is

to underestimate the true correlation between the selection device (e.g., test scores) and first-year

grades (or some other criterion of success) in the original population � that is, the applicant pool.

That range restriction can sometimes have a dramatic effect on the magnitude of validity

coefficients has been clearly demonstrated (e.g., Linn & Dunbar, 1982). Conversely, very high

validity coefficients can be realized when selection is not based on test scores and when,

therefore, the range of performance on the test is not restricted. For instance, Huitema and Stein

(1993) found that when admissions decisions were made without reference to test performance,

GRE General Test scores were reasonably strong predictors of graduate course grades and

faculty ratings, with validity coefficients ranging from .55 to .70. Undergraduate grade point

averages (GPAs), on the other hand, which were used in selection, did not correlate significantly

with these criteria.

Criterion unreliability. A second factor, criterion unreliability, can also serve to decrease

estimates of validity. Because of the attenuation that results from the use of imperfectly reliable

indicators of success, it has often been deemed appropriate to correct for measurement error in

the criteria. Although, historically, this practice has not been accepted unconditionally,

3

correcting for unreliability in the criterion, if not in the predictors, is now generally regarded as

appropriate, provided that both the corrected and the uncorrected results are presented

(AERA/APA/NCME, 1999). The chief remaining debate concerns the proper method of

computing the reliability estimate that is used to make the corrections (Muchinsky, 1996).

For some criteria of success � first-year GPA, for example � the factors that contribute to

imperfect reliability are at least partially understood. For instance, GPAs are often a peculiar

amalgam of individual course grades based on different subject matter, different instructors, and

different grading standards. Such heterogeneity of component parts is known to contribute to the

less-than-perfect reliability of a composite. Combining grades from disparate courses into a

single average often obscures important differences in the meaning of individual course grades,

thereby depressing the reliability of the composite (Willingham, 1990). Grades in single

individual courses, on the other hand, can sometimes be nearly as predictable as composites

based on several courses (Ramist, Lewis, & McCamley, 1990). Accordingly, as discussed below,

one of our objectives was to examine the relationship of both GRE test scores and undergraduate

GPAs to grades in key individual first-year courses.

Compensatory selection. When good standing on one selection variable is allowed to

offset, or �compensate� for, lower status on another, compensatory selection is said to occur.

Like range restriction and criterion unreliability, this circumstance can also have a dramatic

impact on validity estimates. The phenomenon is germane here because compensatory selection

is, very likely, the norm for admission to graduate education generally and veterinary medicine

specifically. Most schools, we believe, follow recommended practice by eschewing rigid cutoffs

with respect to test performance and other admissions criteria. Most likely, perhaps, is the

following scenario described by Cornell (1998). Applicants with relatively low GRE scores may

be selected because they show evidence of other outstanding traits, such as motivation or

maturity. These important qualities may, however, receive less scrutiny for applicants with

exemplary test scores. Subsequently, some students with high test scores may fail because they

lack motivation or maturity, whereas some low-scoring students may succeed largely because

they do possess these qualities.

4

Beyond these anecdotes, there is also strong empirical evidence of the effects of

compensatory selection. Differential compensatory selection practices, as indexed by the extent

to which preadmission measures fail to correlate strongly with one another, can sometimes

explain much of the observed variation among validity coefficients � in the case of law school

admissions, for example, more than 50% (Linn & Hastings, 1984). As Ruscio (1998) has pointed

out, compensatory selection often �� stacks the deck squarely against the predictive validity of

the GRE� (p. 569).

Validity Generalization

Besides acting to depress validity estimates, the factors discussed above can also

contribute to apparent differences in the results of validity studies across situations (so-called

�situational specificity�). Interest in the extent to which the validity of test-score inferences is

similar, or �generalizes,� from one situation to another was, in large part, the motivation for the

development of validity generalization methods. Since the 1970s, these methods have been

adopted in numerous validity studies, most of them concerned with employment testing and

some with academic admissions testing.

The general conclusion from the bulk of these efforts is that the apparent variability in

test score validity across situations or among institutions is often largely artifactual. That is,

differences from one situation to another are more likely the result of statistical phenomena than

of inherent differences among situations in the predictability of criteria. In fact, it is now

commonly accepted that statistical artifacts account for much, if not most, of the apparent

variation among validity estimates across situations. Although a variety of factors, including

computational errors, have been implicated in between-study differences (Hunter & Schmidt,

1990), the bulk of the variation seems due mainly to differences among situations with respect to

the reliability of the criterion, the size of the study samples, and the effects of selection in

curtailing the range of test scores.

For instance, Linn, Harnisch, and Dunbar (1981) estimated that, for a large sample of law

schools, fully 70 percent of the variation in observed validity coefficients could be explained by

these three factors. To a lesser degree, the same result has been found in graduate admissions

5

(Boldt, 1986). (The greater generalization in the law school context is, most likely, due to the

fact that, when compared with graduate departments, law schools are typically more

homogeneous in many respects � for example, with regard to their curricular offerings and their

course requirements.) More recently, Kuncel, Hezlett, and Ones (2000) also concluded that the

relatively low validities observed in GRE validity studies, as well as their variability across

graduate departments, is largely the result of sampling error and range restriction in test scores.

Veterinary Medical School Admissions

Currently, there are 27 U.S. schools of veterinary medicine. As of January 1997, a

majority of these schools required applicants to submit GRE General Test scores, a small number

required scores from the Veterinary College Admission Test (VCAT), and a few others allowed

scores from either the GRE, the VCAT, or the Medical College Admission Test (MCAT)

(Veterinary Medical College Application Service, 1995). Veterinary schools are therefore a

major user of GRE scores.

Befitting this use, a number of veterinary medical professionals have studied the

effectiveness of admissions procedures in their locales, as discussed in the next section.

However, to our knowledge, there has been no large-scale, multi-institution study of the validity

of GRE test scores for veterinary medical school admissions. The significant number of validity

studies that have been conducted by the GRE program on behalf of graduate departments are

relevant, of course (see chapter IX, �Validity of the GRE Tests,� in Briel, O�Neill, &

Scheuneman, 1993). But, the extent to which these studies generalize without reservation to

colleges of veterinary medicine is unclear. On one hand, both graduate education and veterinary

medical education entail challenging post-baccalaureate academic experiences � the kind for

which the GRE General Test was designed. On the other hand, there are some obvious and

meaningful differences between these educational contexts.

6

Relevant Research

As suggested above, there have been several studies of the effectiveness of veterinary

medical admissions (see Appendix for a summary). There is considerable variation among these

studies � for example, with respect to the samples that were studied, the preadmission variables

that were considered, and the criteria of success that were specified. For most of the studies,

samples were pooled across several entering classes within a single institution. Some of the

efforts examined academic performance in the first year of veterinary school, while others

studied performance over a longer period. Various tests were used as predictors, with GRE

scores being considered in about half the studies. The studies span a variety of institutions, with

Oklahoma State University being represented more often than other institutions, due to several

studies by Confer and his colleagues at that institution (Confer, 1990; Confer & Lorenz, in press;

Confer, Turnwald, & Wollenburg, 1995).

Research Questions

The following questions motivated the study described here:

• What is the predictive power � both observed and true (i.e., corrected for factors thattend to lower correlations) � of the GRE General Test and of undergraduate gradeswith respect to measures of success in veterinary medical school? What do GREscores contribute to prediction above and beyond undergraduate grades?

• How much variation is there among veterinary medical colleges with respect to thepredictive validity of GRE scores and undergraduate grades? How much of theapparent variation can be explained by statistical artifacts, such as sampling error,differences in the reliability of criteria, and differential restriction of range in thepredictors and the criteria?

• Is there evidence of compensatory selection (i.e., allowing an applicant�s strength inone area to compensate for a weakness in another) in veterinary school admissions? Ifso, what effect does this have on estimates of test-score validity?

Secondarily, we were also interested in answering the following questions:

• How well do GRE General Test scores and undergraduate grades predict a morestudent-oriented criterion of success in veterinary school (versus the more traditionalinstitution-centered criterion of first-year grades)?

7

• How does the prediction of grades in key individual courses compare with theprediction of overall first-year averages?

Method

Sample Selection

In the fall of 1998, the Association of American Veterinary Medical Colleges (AAVMC)

issued, on our behalf, an invitation to each of the 27 U.S. colleges of veterinary medicine to

participate in a study of the validity of GRE General Test for admissions to colleges of veterinary

medicine. In order to participate, schools were required to provide first-year students� GPAs in

each semester of the 1998-99 academic year. For students who had withdrawn without a GPA,

schools were asked to designate the student�s academic standing at the time of withdrawal (as

either "dismissed for academic reasons" or "withdrew in good standing"). Schools were also

encouraged, but not required, to provide students� grades in individual, key courses of their

choice, and to collect information about students� perceptions of their first-year of veterinary

medical education. The latter was to be accomplished by administering a brief questionnaire

(described in more detail below) to first-year students. Given these requirements, nearly all of the

schools that require or allow applicants to submit GRE scores agreed to cooperate in the study,

resulting in a total of 16 participating schools, which are listed in Table 1.

All but one of the cooperating schools also participate in the AAVMC-sponsored

Veterinary Medical College Application Service (VMCAS). The VMCAS is a central processing

service that receives applications from prospective veterinary school students, conducts analyses

of undergraduate transcripts, assembles and forwards applicants� information to colleges, and,

finally, records admissions decisions made by participating veterinary colleges. For our study,

students� GRE scores, undergraduate GPAs, and demographic characteristics were available

from the VMCAS both for applicants and for admitted students for most schools. Non-VMCAS

schools provided GRE scores, undergraduate GPAs, and other required information for enrolled

students and applicants directly to us.

8

Table 1

Participating Universities

Universities

Iowa State UniversityKansas State UniversityLouisiana State UniversityNorth Carolina State UniversityOhio State UniversityOklahoma State UniversityOregon State UniversityPurdue UniversityTexas A & M UniversityTufts UniversityUniversity of California, DavisUniversity of FloridaUniversity of GeorgiaUniversity of Illinois, UrbanaUniversity of PennsylvaniaWashington State University

The resulting sample of some 1,400 students included more than half of the estimated

2,300 students who entered U.S. veterinary schools in the fall of 1998. In each school, female

students were in the majority and, overall, they comprised about 70% of the total study sample.

With respect to race/ethnicity, the median percentage of Caucasian students across schools was

86%. African American students constituted the next largest group � on average, approximately

1% of the remaining sample. The median percentage of each of 12 other identifiable racial/ethnic

groups was less than 1%. Information on racial/ethnic identity was unavailable for 8% of the

total study sample.

Information was also available for a much larger number of applicants to veterinary

schools � about 5,400 of the approximately 6,600 students who recently submitted some 23,000

applications through the VMCAS (July 1997, http://www.aavmc.org/appdata.htm). Thus, the

study sample comprises a majority of U.S. veterinary schools, first-year veterinary medical

students in these schools, and students who were interested in pursuing veterinary medical study

during the 1998-99 school year.

9

Instruments/Data

Specific information available from the VMCAS or directly from schools included:

• GRE General Test scores1 for most enrolled students and applicants. (Test scoresfrom the Medical College Admissions Test, the Veterinary College Admissions Test,and the GRE Subject Test in biology were also available for some students, but thesedata proved too sparse to conduct any meaningful analyses).

• Undergraduate GPAs (average in last 45 hours of courses, designated below asUGPA45, and overall, designated below as UGPAT). (All GPAs were based on thelatest transcripts available through and including the fall semester of the finalundergraduate year.)

• Schools to which each applicant had applied.

Information obtained from participating veterinary schools included:

• GPAs in each term of the first year of veterinary school

• grades in key individual courses for some schools

• student perceptions of their own first-year success and experiences

The student perceptions questionnaire mentioned earlier was adapted from several

sources, including a recent revision of two Educational Testing Service (ETS®) questionnaires �

the Student Instructional Report (SIR II) and Student Reactions to College � designed to solicit

students' opinions about their college experiences (ETS, 1995). Additional questions were based

on a survey of the academic experiences of graduate management students (Baydar, Dugan,

Grady, & Johnson, 1995). The resulting five-to-10-minute questionnaire was administered to

first-year students shortly after the beginning of their second academic term.

Data Analysis

To answer each of the questions posed at the outset, several analyses were performed.

First, to estimate observed validities, ordinary least-squares regression analyses were conducted

for each school. Three separate criteria (cumulative first-year average, grades in individual

10

courses, and student perceptions of the first year of veterinary school) were regressed, in turn, on

each of several predictors � either individually or in combination. GRE General Test scores

(verbal, quantitative, and analytical) and undergraduate GPAs (both overall and in the last 45

hours of courses) served as the predictors. Zero-order and multiple correlations were computed

for each predictor/criterion combination for each school. In cases where one or more regression

weights were negative for a combination of predictors, the negatively weighted variable was

deleted, and the regressions were re-run for the reduced set of predictors.

Second, to estimate the true (as opposed to observed) validity coefficients for GRE scores

and undergraduate grades, we corrected for both the effects of criterion unreliability and the

influence of range restriction in the predictors. To adjust for the effects of range restriction, we

made use of information about applicants and admitted students at each school. For applicants,

this information included the covariances among predictors; for admitted students, it included the

covariances among predictors and criteria. Using this information, multivariate range restriction

formulas (Gulliksen, 1950, chapter 13) were used to correct both zero-order and multiple

correlations. Multivariate procedures were preferred over univariate methods, because the former

have been shown to yield more accurate, less conservative estimates than the latter (Held &

Foley, 1994; Linn, Harnisch, & Dunbar, 1981). Finally, empirical range restriction procedures

(Linn & Hastings, 1984) were used as an additional check.

Because of both student attrition and school grading practices, range restriction can also

occur on the criterion, as students who perform poorly during their first year of veterinary school

(and who perhaps also had low GRE test scores) may drop out before receiving grades. The

resulting curtailment on the criterion can also act to depress validity estimates. Our approach to

dealing with this type of restriction was to examine the relationship between validity coefficients

and the percentage of nonpersisters at each school. A negative relationship would suggest the

likelihood that validity estimates had been attenuated because of student attrition.

1 More than one set of GRE scores was available for some students. Correlations among highest, most recent, andaverage GRE scores were in the mid- to high .90s for each school. A decision was made to use the most recentscores available for each student.

11

To adjust for the effects of imperfectly reliable criteria, first-year grades were corrected

for attenuation using classical formulas first suggested by Spearman (1904). Our initial intention

was to compute the correlation of grades in a single course in one term with grades in the same

course in a subsequent term, and then to average the resulting correlations over all multi-term

courses. This procedure seemed slightly more appropriate than did internal consistency estimates

based on correlations among grades from heterogeneous courses within a single semester, or

based on correlations between first- and subsequent-term averages, as this method would better

control for differences among courses, both within and across semesters. It would also more

nearly meet the assumptions that individual courses are equally demanding, that they are graded

in an equivalent manner, and that real academic progress (or backsliding) over two semesters is

not plausible.

Although we were able to collect information about performance in individual courses,

the number of multi-term courses did not prove adequate for our purposes. We therefore used

one of the alternative procedures, estimating the reliability of first-year GPAs from the

correlations between first- and second-term averages (and for some schools, third-term averages

as well). The resulting between-term correlations were stepped up according to the Spearman-

Brown prophecy formula in order to estimate the reliability of GPAs for the full first year. To

confirm that these estimates were reasonable, we also computed correlations among individual

course grades at each school, and then stepped up the estimates again using the Spearman-Brown

formula.

Third, to determine how much, if any, of the observed variation among validity

coefficients was due to statistical artifacts, we employed standard meta-analytic/validity

generalization procedures (Hunter & Schmidt, 1990; Pearlman, Schmidt, & Hunter, 1980;

Schmidt, Law, Hunter, Rothstein, Pearlman, & McDaniel, 1993). The artifacts of interest were

small samples, differential restriction of range in the predictors, and differences in criterion

reliability. To begin, we calculated the variation among observed validity coefficients that could

be explained solely by sampling error. The validity coefficients for each individual school were

then corrected both for range restriction and criterion unreliability. Having data from each

participating school, we were able to correct the correlations for each individual school: There

12

was no need to resort to using assumed distributions of artifacts, as is customary in many meta-

analytic studies (Hedges, 1988).

Finally, in order to establish the presence of compensatory selection in veterinary school

admissions (and, if present, to determine its effect on validity estimates), the following strategy

was used. For each school, correlations between predictors � undergraduate grades and GRE

scores, for example � were computed, and correlations for applicants and enrolled students were

compared. When between-predictor correlations were lower for enrolled students than for

applicants, this was taken as evidence that preadmission measures were operating in a

compensatory fashion; the lower the between-predictor correlations among enrolled students, the

greater the degree of compensatory selection was assumed to be.

To assess the likely influence of compensatory selection under these assumptions, the

relation between these correlations and validity coefficients was examined, and the extent to

which the size of these correlations accounted for variation among the size of validity estimates

was calculated. The data were also inspected for indications that selection was based on

preadmission factors other than those for which we had data. For example, we noted the extent to

which applicants with both poor test scores and low undergraduate grades had been admitted.

Results

Results are presented here in terms of each of the questions posed at the outset.

1. What is the observed validity of the GRE General Test, of undergraduate grades, and of thecombination of both for predicting success in each of several veterinary medical schools?

Table 2 displays the results of the regression analyses for each of the 16 participating

veterinary colleges. Zero-order correlations are displayed for individual predictors, and multiple

correlations are shown for combinations of predictors.2 For each predictor and predictor

2 For some of the analyses that involved combinations of predictors, the regression weights computed for one ormore variables were negative. Although none of these negative weights was statistically significant in any of thesecases, variables with negative weights were deleted from the predictor set, and the multiple correlation wasrecomputed for the remaining variables. We note also the existence of possible suppressor effects for some schools �that is, a situation in which one predictor has essentially no predictive power by itself, but which contributes toprediction by suppressing irrelevant variance in another predictor, thus improving the power of that predictor.

13

combination, there is apparent variation among schools. (For one school � a clear outlier � the

computed validities were negative for each of the three GRE General Test scores.) The mean

correlations over all schools were slight to moderate for each individual variable, ranging from

.21 for GRE verbal scores to .37 for undergraduate GPA in the last 45 hours of courses. Overall

undergraduate GPA and undergraduate GPA in the last 45 hours of courses were, on average,

about equally predictive of performance in the first year of veterinary school. Table 2 also shows

that, as has been well established in numerous previous validity studies, the combination of test

scores and previous grades constitutes a better predictive combination than either test scores or

previous grades alone. The mean multiple correlation based on undergraduate GPAs and GRE

General Test scores was .51 when overall undergraduate GPA was used and .53 when

undergraduate GPA in the last 45 hours of courses was available.

2. What are the estimates of the true validity of GRE scores and undergraduate grades aftercorrecting for the effects of criterion unreliability and range restriction in the predictors?

For each school, Table 3 displays � for both applicants and enrolled students � the means

and standard deviations for each predictor variable. As is clear from the differences between

applicants and enrolled students at each school, some selection is taking place � if not on GRE

scores and undergraduate GPAs, then on some other factors that are correlated with them. For

instance, at each school the average overall undergraduate GPA is higher for enrolled students

than for applicants by .5 to 1.1 standard deviation units, and by .4 to 1.0 standard deviation units

for the average undergraduate GPA in the last 45 hours of courses. GRE General Test score

means are also higher for enrolled students than for applicants at each school � by .2 to 1.0

standard deviation units for GRE verbal ability scores, by .1 to 1.0 standard deviation units for

GRE quantitative ability scores, and by 0 to 1.0 standard deviation units for GRE analytical

ability scores. Figure 1 presents distributions of validity coefficients for these three scores, taken

in combination and also combined with overall undergraduate GPA, before and after corrections.

The same combinations are shown in Figure 2 when undergraduate GPA in the last 45 hours of

courses is substituted for overall undergraduate GPA.

In addition to evidence of student selection by institutions, there is also some indication

of student self-selection to veterinary colleges. Although we do not know the mean

undergraduate GPAs of all graduate school applicants, it seems that the means for veterinary

14

school applicants are relatively high, ranging across schools from 3.13 to 3.35 for overall

undergraduate GPA and from 3.25 to 3.45 for undergraduate GPA in the last 45 hours of courses.

Likewise, the mean GRE scores of these applicants tend to be higher than the mean scores of all

GRE General Test takers. During a recent testing year, GRE General Test score means were 474

for GRE verbal ability (SD = 114), 558 for GRE quantitative ability (SD = 139), and 547 for

GRE analytical ability (SD = 130). For GRE test takers who majored in the life sciences (from

which a disproportionate number of veterinary school applicants probably come), the means

were, respectively, 465 (SD = 96), 547 (SD = 117), and 559 (SD = 117; ETS, 1998).

Thus, according to these data, veterinary students are a relatively highly selected group.

The major import of this selection is, for us, its effect on the variation of predictor variables. As

is evident, there is a clear tendency for there to be less variation among enrolled students than

among applicants, hence the need to correct for range restriction.

There is also a need to correct for the imperfect reliability of the primary criterion, first-

year GPA. Reliability estimates based on between-term correlations ranged from .74 to .98

across schools (median = .92). Estimates based on correlations among individual courses ranged

from .81 to .96 (median = .90). Because more data were available, and fewer assumptions

needed, for reliability estimates based on term averages than on individual course grades, the

between-term estimates were used to correct for attenuation due to unreliability.

Table 2

Correlations of Preadmissions Variables with First-Year Average in Veterinary Colleges of Medicine

VariableIndividually In combination

SchoolNumber of

students UGPAT UGPA45 GRE-V GRE-Q GRE-A V,Q,A UGPAT, V,Q,A UGPA45, V,Q,A

A 122 .37 .36 .37 .41 .38 .49 .55 .56B 83 .34 .43 .07 .42 .43 .50a .58a .63a

C 82 .50 .48 .20 .22 .23 .28 .58 .56D 100 .46 .48 .26 .40 .30 .43 .57 .61E 101 .43 .36 .07 .20 .15 .21 .48 .46b

F 97 .33 .33 .18 .20 .13 .24 .49 .43c

G 79 .27 .33 -.12 -.22 -.06 .00d n.c. n.c.H 73 .24 .06 .41 .53 .42 .57 .62 .59I 133 .45 .55 n.a. n.a. n.a. .27e .48 .60J 76 .08 .43 .42 .51 .46 .62 .62 .72K 36 .11 .26 .44 .09 .16 .46c .47c .56c

L 108 .24 .16 .20 .19 .15 .24 .39 .32M 53 .37 .39 .27 .31 .24 .34 .42 .46N 130 .32 .49 .12 .27 .21 .27 .41 .58O 77 .44 n.a. .27 .08 .35 .41c .57 n.a.P 70 .38 .25 .19 .20 .25 .29 .44 .35

Median 83 .36 .36 .20 .22 .24 .32 .49 .56Weighted mean 89 .35 .37 .21 .26 .25 .34 .51 .53

Note. UGPAT = overall undergraduate grade point average; UGPA45 = undergraduate grade point average in last 45 hours; GRE-V = GRE General Testverbal ability score; GRE-Q = GRE General Test quantitative ability score; GRE-A = GRE General Test analytical ability score; V,Q,A = combinedGRE General Test verbal ability, quantitative ability, and analytical ability score; n.a. = not available; n.c. = not computed.a Does not include GRE-V; b does not include GRE-A; c does not include GRE-Q; d set to .00 because all weights were negative; e only a total of GRE-V,GRE-Q, and GRE-A was available for this school.

15

16

Table 3

Means and Standard Deviations of Preadmissions Variables for Applicants andEnrolled Students

VariableSchool UGPAT UGPA45 GRE-V GRE-Q GRE-A

A Applicants 3.21 (.40) 3.32 (.42) 504 (94) 600 (95) 609 (105)Enrollees 3.52 (.30) 3.64 (.27) 591 (89) 697 (60) 713 (66)

B Applicants 3.17 (.38) 3.27 (.42) 480 (87) 580 (93) 580 (115)Enrollees 3.41 (.29) 3.46 (.34) 523 (72) 655 (72) 615 (119)

C Applicants 3.16 (.37) 3.26 (.42) 476 (89) 561 (93) 571 (110)Enrollees 3.43 (.33) 3.58 (.32) 569 (82) 649 (61) 650 (84)

D Applicants 3.20 (.37) 3.32 (.40) 487 (88) 580 (98) 586 (105)Enrollees 3.58 (.24) 3.65 (.26) 516 (80) 628 (72) 648 (84)

E Applicants 3.18 (.38) 3.30 (.41) 476 (89) 574 (95) 580 (107)Enrollees 3.53 (.24) 3.58 (.22) 500 (77) 612 (78) 641 (82)

F Applicants 3.19 (.36) 3.32 (.39) 476 (86) 574 (92) 579 (106)Enrollees 3.42 (.17) 3.50 (.25) 512 (92) 622 (69) 656 (94)

G Applicants 3.13 (.38) 3.26 (.41) 464 (85) 560 (92) 566 (111)Enrollees 3.55 (.26) 3.66 (.23) 491 (83) 584 (69) 566 (130)

H Applicants 3.23 (.38) 3.34 (.42) 486 (88) 585 (93) 594 (104)Enrollees 3.62 (.28) 3.71 (.36) 530 (89) 650 (74) 664 (78)

I Applicants 3.35 (.36) 3.45 (.37) 495 (91) 590 (94) 600 (105)Enrollees 3.57 (.27) 3.73 (.19) n.a. n.a. n.a.

J Applicants 3.14 (.38) 3.25 (.41) 463 (83) 553 (94) 569 (107)Enrollees 3.40 (.28) 3.52 (.24) 480 (79) 567 (100) 593 (99)

K Applicants 3.20 (.38) 3.31 (.42) 488 (90) 582 (95) 592 (105)Enrollees 3.60 (.25) 3.71 (.22) 568 (82) 636 (88) 669 (69)

L Applicants 3.26 (.38) 3.39 (.39) 509 (92) 601 (93) 610 (103Enrollees 3.45 (.29) 3.56 (.28) 568 (88) 664 (71) 651 (93)

M Applicants 3.17 (.39) 3.27 (.42) 477 (86) 582 (94) 586 (106)Enrollees 3.48 (.30) 3.62 (.25) 505 (96) 630 (80) 621 (80)

N Applicants 3.20 (.38) 3.31 (.41) 470 (91) 574 (93) 582 (110)Enrollees 3.61 (.25) 3.69 (.21) 528 (95) 647 (78) 657 (87)

O Applicants n.a. n.a. n.a. n.a. n.a.Enrollees 3.45 (.31) n.a. 599 (79) 675 (63) 692 (69)

P Applicants 3.23 (.37) 3.34 (.40) 490 (91) 585 (92) 595 (104)Enrollees 3.55 (.24) 3.69 (.18) 508 (84) 602 (65) 635 (92)

Median Applicants 3.20 3.31 480 580 586Enrollees 3.52 3.64 523 636 650

Note. For applicants, undergraduate GPA statistics are based on from 629 to 1550 applicants per institution;GRE General Test scores are based on from 885 to 2356 applicants per institution; n.a. = not available.

17

Figure 1

Distributions of Validity Coefficients Before and After Corrections

(UGPAT as the Undergraduate Grade Indicator)

1515 1616 1616N =

CorrectedUncorrected

Valid

ity C

oeffi

cien

ts

1.0

.8

.6

.4

.2

0.0

UGPA Total

GRE V,Q,A

UGPA Total

& GRE V.Q,A

Figure 1. Distributions of validity coefficients before and after corrections (UGPAT as

the undergraduate grade indicator).

18

1414 1616 1515N =

CorrectedUncorrected

Valid

ity C

oeffi

cien

ts

1.0

.8

.6

.4

.2

0.0

UGPA Last 45 Hrs

GRE V,Q,A

UGPA Last 45 Hrs

& GRE V.Q,A

Figure 2. Distributions of validity coefficients before and after corrections (UGPA45 as

the undergraduate grade indicator).

19

Correcting back to each school�s applicant pool for multivariate range restriction resulted

in somewhat higher validity estimates for each school and for each predictor combination (Table

4). Median validity estimates increased by .16 to .24 for individual predictors, and by .17 to .21

for combinations of predictors. When corrections were applied for both range restriction and

criterion unreliability (Table 5), correlations increased slightly more. The increase in median

correlations was .01 to .05 for individual predictors and .02 to .04 for combination of predictors

when correlations were corrected for attenuation due to criterion unreliability. The relatively

small size of this additional increase was expected because first-year GPA was reasonably

reliable, on average, across schools. Figures 1 and 2 display distributions of uncorrected and

corrected validity coefficients for overall undergraduate GPA alone, for GRE verbal,

quantitative, and analytical scores in combination, and for undergraduate GPA in the last 45

hours of courses.

Table 4

Correlations (Corrected for Range Restriction Only) of Preadmissions Variables With First-Year Average in VeterinaryColleges of Medicine


SchoolNumber of

students UGPAT UGPA45 GRE-V GRE-Q GRE-A V,Q,A UGPAT, V,Q,A UGPA45, V,Q,A

A 122 .50 .52 .50 .55 .56 .64 .70 .72B 83 .47 .55 .28 .56 .51 .60a .67a .71a

C 82 .62 .59 .38 .48 .44 .53 .71 .70D 100 .62 .67 .40 .54 .50 .59 .74 .78E 101 .70 .69 .36 .44 .33 .47 .75 .74b

F 97 .79 .73 .31 .34 .43 .45 .85 .80c

G 79 .39 .45 n.c. n.c. n.c. .00d n.c. n.c.H 73 .48 .43 .54 .61 .51 .68 .74 .72I 133 .69 .80 n.a. n.a. n.a. .49e .73 .83J 76 .39 .63 .48 .53 .49 .62 .66 .80K 36 .40 .62 .57 .34 .38 .59c .64c .76c

L 108 .47 .37 .38 .39 .31 .45 .56 .51M 53 .49 .55 .29 .35 .30 .39 .54 .60N 130 .62 .78 .30 .43 .34 .45 .67 .81O 77 .56 n.a. .47 .49 .56 .60c .73 n.a.P 70 .53 .38 .30 .34 .34 .40 .58 .49


Note. Correlations are corrected for range restriction but not for criterion unreliability. UGPAT = overall undergraduate grade point average; UGPA45 = undergraduategrade point average in last 45 hours; GRE-V = GRE General Test verbal ability score; GRE-Q = GRE General Test quantitative ability score; GRE-A = GRE GeneralTest analytical ability score; V,Q,A = combined GRE General Test verbal ability, quantitative ability, and analytical ability score; n.a. = not available; n.c. = notcomputed.a Does not include GRE-V; b does not include GRE-A; c does not include GRE-Q; d set to .00 because all weights were negative; e only a total of GRE-V, GRE-Q, andGRE-A was available for this school.

20

Table 5

Correlations (Corrected for both Range Restriction and Criterion Unreliability) of Preadmissions Variables With First-YearAverage in Veterinary Colleges of Medicine


SchoolNumber ofStudents UGPAT UGPA45 GRE-V GRE-Q GRE-A V,Q,A UGPAT, V,Q,A UGPA45, V,Q,A

A 122 .52 .54 .52 .57 .58 .66 .72 .74B 83 .55 .64 .33 .65 .59 .70a .78a .82a

C 82 .63 .60 .39 .49 .45 .54 .72 .71D 100 .65 .70 .42 .56 .52 .62 .77 .81E 101 .71 .70 .37 .45 .34 .48 .76 .75b

F 97 .85 .79 .33 .37 .46 .49 .92 .86c

G 79 .43 .49 n.c. n.c. n.c. .00d n.c. n.c.H 73 .51 .45 .57 .64 .54 .72 .78 .76I 133 .70 .81 n.a. n.a. n.a. .49e .74 .84J 76 .42 .68 .52 .57 .53 .67 .71 .86K 36 .42 .65 .59 .35 .40 .62c .67c .79c

L 108 .55 .43 .44 .45 .36 .52 .65 .59M 53 .51 .58 .30 .37 .31 .41 .57 .63N 130 .64 .80 .31 .44 .35 .46 .69 .84O 77 .57 n.a. .48 .50 .57 .62c .75 n.a.P 70 .57 .41 .32 .36 .36 .43 .62 .53


Note. Correlations are corrected for both range restriction and criterion unreliability. UGPAT = overall undergraduate grade point average; UGPA45 =undergraduate grade point average in last 45 hours; GRE-V = GRE General Test verbal ability score; GRE-Q = GRE General Test quantitative ability score; GRE-A = GRE General Test analytical ability score; V,Q,A = combined GRE General Test verbal ability, quantitative ability, and analytical ability score; n.a. = notavailable; n.c. = not computed.a Does not include GRE-V; b does not include GRE-A; c does not include GRE-Q; d set to .00 because all weights were negative; e only a total of GRE-V, GRE-Q,and GRE-A was available for this school.

21

22

To account for the possible role of student attrition in restricting the range of performance

on the criterion (and perhaps the predictors), we asked schools to indicate which students had

withdrawn, and why � that is, whether they had withdrawn in good standing or had been

dismissed for poor academic performance. Twelve schools were able to provide this information,

indicating in each case that, in total, five or fewer students had withdrawn in either good or poor

academic standing. Thus, attrition exerted little, if any, influence on the accuracy of validity

estimates.

In summary, when correlations were fully corrected for both range restriction in the

predictors and unreliability in the criterion, the resulting validity coefficients were moderately

large. Median correlations were in the .40s for each component of the GRE General Test and .53

for the combination of all three GRE General Test scores. Undergraduate GPA proved to be an

even stronger predictor, with median correlations of .56 for overall undergraduate GPA and .64

for undergraduate GPA in the last 45 hours of courses. Taken together, GRE scores and

undergraduate GPA constitute a relatively powerful predictive combination, accounting, on

average, for slightly more than half the variance in first-year veterinary school GPAs. By

considering GRE scores along with undergraduate GPA, the amount of variance explained

increases by about 18% � from 35-40% to 53-58%. According to Cohen�s guidelines for effect

size (1977, chapter 9), this increase in explained variance can be regarded as �medium� to

�large.�

3. How much of the apparent variation among validity coefficients across schools can beexplained by statistical artifacts such as (a) small samples, (b) differential restriction of range inthe predictors, and (c) differences in criterion reliability?

To partition the variation among validity coefficients into that due to random fluctuation

versus that due to systematic differences among schools, meta-analytic formulae (Hunter &

Schmidt, 1990, chapter 3) were applied � first to the uncorrected validity coefficients given in

Table 2 and then to the fully corrected coefficients in Table 5. The observed variation among

validity coefficients across schools, and the proportion attributable solely to sampling error, was

as follows for individual predictors and combinations:

23

Predictor Observed variance Variance due to sampling Percentage

UGPAT .0112 .0088 79%

UGPA45 .0167 .0084 50%

V,Q,A .0154 .0089 58%

UGPAT , V,Q,A .0055 .0062 100%

UGPA45, V,Q,A .0114 .0058 51%

For the fully corrected correlations, the variance of population correlations was estimated

by using the appropriate formula for such corrections (Hunter & Schmidt, 1990, pp.145-148).

For (a) overall undergraduate GPA alone, (b) GRE verbal, quantitative, and analytical scores in

combination, and (c) overall undergraduate GPA and all three GRE scores in combination, the

estimated variance of population correlations was slightly negative, suggesting little if any

variation among true validity coefficients across schools. For the combination of undergraduate

GPA in the last 45 hours of courses and all three GRE scores, the variance of the population

correlations was slight (.0025). Thus, it appears that after correcting for range restriction and

criterion unreliability, most of the remaining variation can be attributed to sampling error.

4. Is there evidence of compensatory selection in veterinary school admissions? If so, how doessuch selection affect estimates of the predictive validity of test scores?

Although the data were insufficient to provide a definitive answer to this question, it

seemed useful at least to explore this issue. An inspection of the distributions of preadmission

variables for enrolled students at each school revealed the presence of one or more students with

relatively low standing on at least one variable, suggesting, at the least, that schools did not

simply apply multiple absolute cutoffs below which applicants were automatically disqualified.

For instance, with respect to GRE verbal ability scores, the lowest score ranged from 240 at one

school to 400 at another (mean of the lowest GRE verbal score across all schools = 347); for

GRE quantitative ability scores, the lowest score ranged from 330 to 530 across schools (mean =

450); and for GRE analytical ability scores , the lowest ranged from 200 to 510 (mean = 380).

For overall undergraduate GPA, the range of low values across schools was from 2.37 to 3.09

(mean = 2.78), and for undergraduate GPA in the last 45 hours of courses, the range was from

1.33 to 3.29 (mean = 2.84).

24

Numerous specific cases suggested the possible operation of compensatory selection

procedures. These include individuals with very high GRE scores and lackluster undergraduate

grades, and vice versa. There were also cases of individuals with relatively poor standing with

respect to both GRE scores and undergraduate grades, suggesting that other factors may have

played a compensatory role in the acceptance of these latter students. In general, the veterinary

schools in our sample seem not to have used strict cutoffs on any particular measure, below

which applicants were automatically rejected. In other words, it is likely that, at least for the

preadmission variables that we have considered, very good standing on one measure was allowed

to compensate for poor standing on another.

A second indication of compensatory selection comes from a comparison of the

correlations among predictors for applicants versus enrolled students at each school (Table 6). As

is clear, the between-predictor correlations are almost always lower for enrolled students than for

applicants. For applicants, the intercorrelations are always positive; for enrolled students, they

are in some cases negative. In part, the lower intercorrelations for enrolled students may be the

result of range restriction due to selection on the basis of individual predictors. They are

probably also a function, however, of compensatory selection in which applicants were admitted

because of good standing on one, but not both, predictors. The effect of compensatory selection

is often to change the joint distribution of two predictors to the point of reversing the sign of their

correlation with one another (see Linn & Dunbar, 1982, for a compelling example).

Table 6

Correlations Among Preadmissions Variables for Applicants and Enrolled Students

School Student GroupGRE-V with

GRE-QGRE-V with

GRE-AGRE-V with

UGPAT

GRE-V withUGPA45

GRE-Q withGRE-A

GRE-Q withUGPAT

GRE-Q withUGPA45

GRE-A withUGPAT

GRE-A withUGPA45

UGPAT withUGPA45

A Applicants .50 .53 .28 .28 .61 .35 .30 .28 .26 .80Enrollees .38 .40 .11 .19 .48 .41 .27 .18 .06 .75

B Applicants .46 .50 .23 .22 .57 .34 .31 .24 .24 .80Enrollees .23 .44 -.13 -.12 .42 .19 .08 .04 .09 .72

C Applicants .46 .52 .25 .23 .58 .28 .26 .24 .20 .78Enrollees .36 .35 .04 .02 .41 -.05 -.06 -.01 .02 .79

D Applicants .48 .49 .25 .25 .60 .34 .32 .28 .26 .79Enrollees .42 .24 .18 .11 .40 .21 .17 .14 -.03 .72

E Applicants .47 .48 .24 .23 .59 .32 .28 .25 .21 .79Enrollees .14 .15 -.17 -.17 .54 .06 -.18 -.01 .02 .59

F Applicants .44 .49 .15 .16 .57 .22 .20 .17 .14 .75Enrollees .29 .35 -.11 -.06 .51 -.16 .14 -.50 -.35 .40

G Applicants .48 .50 .18 .18 .56 .24 .21 .18 .13 .78Enrollees .44 .38 -.12 -.23 .37 -.14 -.34 .19 .08 .71

H Applicants .48 .52 .20 .22 .59 .27 .25 .27 .24 .78Enrollees .43 .35 -.24 -.30 .59 .10 -.06 .16 -.02 .55

I Applicants .48 .50 .29 .29 .62 .36 .30 .34 .29 .82Enrollees n.a. n.a. n.a. n.a. n.a. n.a. n.a. n.a. n.a. .73

J Applicants .42 .51 .19 .19 .55 .23 .19 .15 .14 .75Enrollees .23 .34 .17 .21 .52 .09 .07 -.07 -.02 .57

K Applicants .46 .48 .21 .24 .60 .30 .29 .28 .27 .79Enrollees .10 .10 -.01 -.16 .48 -.06 .07 -.07 .07 .71

L Applicants .50 .48 .25 .25 .62 .33 .29 .24 .22 .82Enrollees .32 .41 -.21 -.15 .52 -.13 -.10 -.15 -.12 .78

M Applicants .45 .48 .26 .22 .57 .35 .28 .24 .20 .80Enrollees .44 .43 .34 .19 .52 .44 .25 .33 .14 .71

N Applicants .48 .52 .23 .19 .58 .33 .26 .24 .18 .77Enrollees .31 .45 -.14 -.10 .57 .11 -.03 .05 -.08 .63

O Applicants n.a. n.a. n.a. n.a. n.a. n.a. n.a. n.a. n.a. n.a.Enrollees .06 .17 .11 n.a. .42 -.27 n.a. .10 n.a. n.a.

P Applicants .45 .50 .21 .23 .59 .30 .26 .25 .23 .79Enrollees .22 .33 .06 .24 .56 .11 .14 .16 .05 .68

Note. n.a. = not available

25

26

The correlations shown in Table 6 are, therefore, clearly consistent with the use of

compensatory selection procedures. They suggest, for instance, that a high GRE score may offset

low undergraduate grades, and vice versa, and that a high score on one component of the GRE

General Test may counteract a lower score on another section. Though not addressed by our

data, it seems plausible that other, unconsidered preadmission information may also act in a

compensatory manner, thus serving to lower the apparent validity of both GRE scores and

undergraduate grade averages. To reiterate Ruscio�s (1998) claim, compensatory selection can

make any given predictor appear less useful than it really is.

Unlike univariate procedures, multivariate range restriction corrections take into account

relationships among all predictors. Therefore, their application may also help to correct for the

downward effect of compensatory selection on estimates of test validity (Charles Lewis, personal

communication, June 22, 2000). To check this possibility, we computed the relationship

between validity coefficients and the correlation between predictors � that is, GRE scores and

overall undergraduate GPA �for both individual predictors and for combinations of them. The

assumption was that, although observed validity might relate, artifactually, to the correlations

among the predictors, true validity should not.

An inspection of these relations revealed that, across schools, the observed validity of

each of the three components of the GRE General Test was related to the GRE-overall

undergraduate GPA correlation (r = .20 to .62), thus suggesting the influence of compensatory

selection on validity estimates. After correcting for range restriction, these correlations decreased

(r = .01 to .24), suggesting that the corrected estimates were less influenced by the effects of

compensatory selection. A similar result was noted for the validity (multiple correlation) of the

combination of GRE scores. The correlation of observed validities with GRE-overall

undergraduate GPA correlations ranged from .10 to .38 before corrections were applied, and

from -.11 to .24 afterwards.

The result of applying range restriction corrections to overall undergraduate GPA � when

used either alone or in combination with GRE scores � was also to decrease the correlations

between validity coefficients and GRE-overall undergraduate GPA correlations. However, the

decreases were large enough to turn positive relationships into negative ones, suggesting the

27

possibility of overcorrection. For instance, for overall undergraduate GPA used alone, the

correlations decreased from .12 to .24 before corrections were applied, to -.13 to -.45 afterwards.

We are reluctant, however, to place too much emphasis on these empirical �checks,� since with

our small sample (n = 16) these correlations were not statistically significant.

5. How well do GRE test scores and undergraduate GPAs predict a more student-orientedcriterion of success in veterinary school (versus the more traditional institution-centeredcriterion of first-year grades)?

To explore this question, we asked students at each school to complete the Student

Perceptions Questionnaire. Students at each of 14 schools responded to the questionnaire, and a

composite score on the questionnaire was computed for each respondent. The estimated

reliability (coefficient alpha) of the composite score was .82. The median correlation (over

schools) of the composite score with overall undergraduate GPA was .03; the median correlation

with total GRE General Test score was .05. In both cases, correlations were nearly as likely to be

negative as positive. Thus, student perceptions of their experiences in the first year of veterinary

school, at least as we have defined them here, were not predictable from either GRE scores or

undergraduate grades. It is informative, however, from the standpoint of providing additional

context for this study, to examine the perceptions of the veterinary medical students included in

the study.

Table 7, which displays student responses to each of the 10 questionnaire items, shows,

for example, that most students believed that they were �learning a lot in their first year of

veterinary school,� and they were generally �satisfied with their progress in learning the

knowledge and skills needed for veterinary medicine.� Reaction was more mixed, however, with

regard to, for example, whether coursework was excessively demanding and whether the pace of

instruction was appropriate. In summary, although student perceptions varied, the general picture

is that the first year of veterinary medicine, though challenging for most students, is not

overwhelming.

28

Table 7

Student Perceptions of the First Year of Veterinary Medical School

School

Perceptions ResponseA

(n = 86)C

(n = 23)D

(n = 87)E

(n = 89)F

(n = 91)G

(n = 60)H

(n = 65)I

(n = 102)J

(n = 25)K

(n = 25)L

(n = 42)M

(n = 53)N

(n = 90)P

(n = 35) MedianSA 73 83 82 81 74 72 79 69 68 96 69 83 72 69 74A 27 17 16 17 25 23 22 28 28 4 29 17 27 31 24N 0 0 2 2 1 3 0 4 4 0 0 0 0 0 0D 0 0 0 0 0 2 0 0 0 0 0 0 1 0 0

1. I�m learning a lotmy first year ofveterinary school.

SD 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0SA 4 30 17 7 3 15 11 25 0 4 7 9 17 6 8A 25 44 38 39 40 30 47 45 20 50 33 47 49 31 40N 35 0 29 33 33 33 28 15 32 8 21 28 20 37 28D 29 26 16 21 22 15 13 13 44 33 31 13 12 17 19

2. I�ve felt over-whelmed by thecoursework duringmy first year.

SD 7 0 0 0 2 7 2 3 4 4 7 2 2 9 3SA 13 9 9 5 3 13 12 9 4 28 10 0 9 3 9A 71 61 62 66 58 57 71 54 64 64 74 34 56 51 63N 14 13 20 18 23 20 15 21 32 8 12 43 28 34 20D 1 17 8 10 13 8 2 15 0 0 5 21 7 11 8

3. My professors�instruction hasusually been clearand understandable.

SD 1 0 1 1 2 2 0 1 0 0 0 2 1 0 1SA 0 5 5 0 0 10 5 18 4 4 2 2 6 3 5A 16 32 28 24 15 25 20 34 32 12 14 23 21 17 24N 39 14 37 47 48 33 50 27 28 44 36 47 52 46 41D 42 50 31 29 35 27 23 20 36 36 45 28 21 31 31

4. The pace atwhich materialhas been coveredhas been too fast.

SD 4 0 0 0 1 5 2 2 0 4 2 0 0 3 1SA 0 0 0 0 0 0 0 1 0 0 2 2 1 0 0A 5 0 5 5 2 5 5 10 0 0 0 2 2 0 2N 16 18 25 19 20 10 22 15 20 16 24 29 19 31 20D 55 68 56 66 70 68 69 62 64 56 55 50 64 49 63

5. For my ability(or level of prep.),the courses haveseemed toodifficult. SD 24 14 14 10 8 17 5 13 16 28 19 17 13 20 15

Note. SA = strongly agree; A = agree; N = neither agree nor disagree; D = disagree; SD = strongly disagree.

Table 7 (continued)

School

Perceptions ResponseA

(n = 86)C

(n = 23)D

(n = 87)E

(n = 89)F

(n = 91)G

(n = 60)H

(n = 65)I

(n = 102)J

(n = 25)K

(n = 25)L

(n = 42)M

(n = 53)N

(n = 90)P

(n = 35) MedianSA 2 4 5 0 0 0 3 6 0 0 5 6 4 3 3A 7 44 20 9 8 8 20 29 12 16 12 17 21 17 16N 31 22 24 33 32 32 37 29 28 24 20 40 37 34 31D 52 30 47 49 54 47 39 32 52 56 51 38 32 43 47

6. The first year hasbeen too stressful.

SD 7 0 5 9 7 13 2 5 8 4 12 0 6 3 6SA 0 4 0 0 0 0 0 2 0 0 2 2 0 0 0A 2 13 8 1 1 3 2 18 8 12 2 8 9 3 5N 17 26 23 18 25 27 42 21 16 12 17 37 27 31 24D 69 57 59 72 66 56 55 52 68 64 67 48 58 57 59

7. The first yearacademic require-ments have been toodemanding.

SD 12 0 10 9 8 14 2 7 8 12 12 6 7 9 9SA 11 13 6 6 3 17 2 9 8 8 10 9 9 9 9A 51 44 43 51 42 44 48 47 48 60 55 32 48 54 48N 31 30 29 36 33 24 20 23 36 8 21 32 23 29 29D 5 9 22 7 21 12 28 21 8 24 10 25 18 9 15

8. I�ve hadrelatively littledifficulty under-standing coursematerial. SD 2 4 1 1 1 3 3 1 0 0 5 2 2 0 1

SA 2 17 13 2 4 7 5 19 16 12 5 19 19 6 9A 17 52 32 32 32 32 39 39 24 24 24 32 36 43 32N 49 13 26 33 36 33 34 23 36 32 24 34 26 37 33D 28 17 26 34 26 22 23 17 24 32 44 15 19 14 24

9. The demandson my time andenergy havebeen excessive.

SD 4 0 2 0 1 7 0 3 0 0 2 0 1 0 1SA 23 22 24 17 31 25 15 12 8 40 22 17 18 3 20A 64 61 56 66 56 62 75 62 64 52 68 62 67 63 63N 9 9 15 10 9 12 8 17 24 8 10 13 10 34 10D 2 9 3 5 4 2 2 9 4 0 0 6 4 0 4

10. I�m satisfied withmy progress in learn-ing the knowledgeand skills needed fora veterinary medicaldegree.

SD 1 0 1 2 0 0 0 1 0 0 0 2 1 0 0

Note. SA = strongly agree; A = agree; N = neither agree nor disagree; D = disagree; SD = strongly disagree.

29

30

6. How predictable are grades in individual core courses versus first-year averages?

Each participating school provided student grades for at least several individual courses

of their choosing. Table 8 displays for each school the highest and lowest correlations observed

between grades in these individual first-year courses3 and GRE scores, as well as undergraduate

GPAs. Also shown for each school is the mean grade for the course in which students received

the highest average grade and for the course in which they received the lowest average grade.

On average, individual course grades were less predictable than were first-year averages.

However, there was clearly significant variability among courses with respect to the

predictability of grades in individual courses. While performance in some courses bore only a

slight, or even negative, relationship with predictors, others were more predictable than overall

first-year average.

Within schools, courses also differed � often substantially � with respect to the average

grades that were awarded: In some courses, a B- or C+ grade was the norm, while in others an A

was apparently the lowest grade awarded. The great variability among individual courses � both

in the average level of grades awarded and in their predictability � is not surprising, perhaps, as

the demands of first-year veterinary medical courses probably vary widely from course to course.

Courses in such diverse subjects as biostatistics, neuroscience, veterinary ethics, and wildlife

management would appear, at least on the face of it, to differ significantly with respect to both

the knowledge and skills required and the specific academic challenges presented.

3 We have not applied range restriction corrections to the correlations for individual course grades, nor have weadjusted these correlations to simulate a more reliable criterion based on a full course load.

31

Table 8

Statistics for Individual Course Grades

Mean grades inindividual courses Range of correlation of course grades with:

SchoolNumber

of courses Lowest Highest UGPAT UGPA45 GRE-V GRE-Q GRE-A

A 16 2.87 3.86 Lowa .03 -.02 .04 .11 .07High .43 .23 .40 .45 .45

B 13 2.79 3.93 Low .09 .11 -.18 .06 .09High .33 .44 .26 .40 .48

C 13 2.52 3.83 Low -.05 -.12 -.16 -.15 -.16High .48 .37 .28 .29 .31

D 9 2.93 3.74 Low .21 .23 .07 .13 .07High .42 .48 .36 .38 .37

E 6 3.03 4.00 Low .29 .22 -.10 .07 .02High .38 .51 .11 .20 .17

F 9 3.01 3.75 Low .05 .01 -.03 -.03 -.05High .45 .65 .41 .26 .21

G 15 2.67 4.00 Low -.06 .02 -.30 -.35 -.24High .31 .38 .35 .23 .30

H 11 3.11 3.99 Low -.04 -.04 .04 .07 -.08High .32 .19 .49 .53 .45

I 16 2.74 3.86 Low .13 .02 � .05b �High .40 .56 � .43b �

J 10 2.60 4.00 Low .00 .08 .17 .12 .21High .17 .31 .39 .52 .41

K 13 3.00 3.97 Low -.10 -.01 .00 -.06 -.11High .23 .44 .52 .20 .34

L 16 3.00 3.98 Low -.17 -.15 -.06 -.24 -.14High .41 .22 .26 .30 .30

M 14 2.70 3.92 Low .04 -.13 -.10 -.20 -.09High .43 .05 .42 .41 .23

N 8 2.56 3.62 Low .15 .26 -.12 .13 .01High .34 .45 .28 .34 .32

O 13 2.91 3.87 Low .11 n.a. .11 -.21 .03High .43 n.a. .33 .18 .42

P 7 2.94 3.66 Low .22 .09 .07 -.06 -.04High .44 .35 .30 .23 .30

Note. n.a. = not availablea �Low� refers to the lowest correlation computed between an individual course grade and the predictor; �high� refersto the highest such correlation.b For a composite of GRE verbal, quantitative, and analytical scores

32

Discussion

It is now commonly acknowledged that the meaning and use of test scores, not the test

itself, is the proper focus of validation efforts (AERA/APA/NCME, 1999; Messick, 1989). An

assessment may, therefore, yield valid score inferences for some purposes and in some contexts,

but not in others. Individual validity studies, such as the one conducted recently by Sternberg and

Williams (1997), can often be useful for generalizing to other similar situations. However, there

are clearly limits to relying on single, local studies to draw conclusions about the validity of test

scores more generally: The greater the difference in context between situations, the greater the

inferential leap is likely to be.

By focusing on a particular context � veterinary medical school admissions � we were

able to secure the cooperation of a majority of U.S. veterinary colleges of medicine and, hence,

to obtain information about a preponderance of first-year veterinary medical students in the

1998-99 academic year. Thus, unlike the results of validity studies based on single institutions,

our findings are, we believe, reasonably representative of a specific universe of interest � in this

case, veterinary medical schools.

As anticipated, the highly selective nature of veterinary medical school admissions was

clearly apparent. As a result, there was ample evidence of factors that can result in seriously

underestimating the validity of preadmission measures. Institutional selection � both general and

compensatory in nature � was clearly apparent, resulting in a restricted range of test scores and

undergraduate grade averages for enrolled students at each school. Even with the dampening

effects of selection, however, both undergraduate GPA and GRE General Test scores exhibited

significant relationships with first-year grades in veterinary medicine. Moreover, when used

together, grades and test scores constituted a more powerful predictor combination than did

either one used alone.

More importantly, when statistical corrections were made to counteract the attenuating

effects of selection, validity estimates increased significantly. They increased further, but

relatively slightly, when corrections were applied to adjust for the unreliability of first-year

veterinary school grade averages. When fully corrected for both range restriction in the

33

predictors and unreliability in the criterion, the combination of undergraduate GPA and GRE

General Test scores accounted for, on average, more than half the variation in first-year

veterinary school GPAs. GRE scores accounted for a �large� portion of variance above and

beyond that explained by undergraduate grades. An analysis of the variation among schools

suggested little reason to question the generalizability of our findings across veterinary medical

schools: With the exception of one school, validity estimates seemed to apply about equally well

to all of the schools that participated in the study.

The uncorrected validities estimated here are relatively similar in size to those reported

previously for graduate departments of arts and sciences. For instance, the median multiple

correlation of first-year veterinary school GPA with the combination of GRE verbal,

quantitative, and analytical ability scores (.32 in the present study) is only slightly higher than

the values (.20 to .30) reported elsewhere for graduate departments (Briel, O�Neill, &

Scheuneman, 1993, p.77). The same is true for undergraduate grades (median r = .36 in the

present study, versus the estimates of .30 to .40 reported by Briel, O�Neill, & Scheuneman).

Finally, because of the aforementioned problems with using first-year GPA as a criterion

of success, we explored the potential of individual course grades as a criterion. This exploration

suggested that further studies utilizing a criterion of this nature would be informative. It also

provided a better sense of why first-year grade averages are sometimes not very predictable.

Furthermore, the average course grade was highly variable across courses and schools,

suggesting the operation of different grading criteria or standards. In addition, individual course

grades also varied substantially with respect to their relationship to GRE scores and

undergraduate grades. Given the considerable heterogeneity of individual course grades, it is

perhaps surprising that a composite based on them was as predictable as it was.

One possibility for further study involving course grades would be a systematic

investigation of the predictability of performance in individual courses, classified according to

such factors as content, cognitive requirements, and stringency of grading standards. Unbundling

first-year grades in this manner, and searching for differential relations with GRE scores, could

further our understanding of the construct validity of GRE General Test scores. Our suggestion,

34

therefore, is to explore the application of such methods as those used by Ramist, Lewis, &

McCamley-Jenkins (1994) to correct for differential grading standards among individual courses.

Limitations

Although we believe that our study successfully addressed several shortcomings that are

typical of many test validation studies, ours is not free from all limitations. We are relatively

confident that the study results represent veterinary medical colleges reasonably well, but we are

less certain about the applicability of our findings to graduate departments. For instance,

although they may have some similarities, the specific academic demands of veterinary medical

education and those characteristic of graduate education undoubtedly differ in at least some

important ways. However, insofar as both contexts present rigorous post-baccalaureate academic

challenges, information about the utility of the GRE General Test and undergraduate grades for

predicting success in veterinary school admissions should be at least somewhat relevant to

graduate school admissions.

A second limitation of the study was the unavailability of some potentially relevant

preadmissions information. Although we were able to tap several sources of data that are often

not available to validity researchers, we did not have access to all possibly pertinent information.

For instance, we were able to obtain information about performance on several of the most

important, traditional preadmission measures � undergraduate grades and admission test scores �

for applicants as well as for enrolled students. We did not, however, have access to a variety of

other information sources � letters of recommendation and personal statements, for instance �

that are used in veterinary school admissions. We believe, however, that because of

compensatory selection practices, the most likely effect of our inability to model admissions

procedures completely was to underestimate the validity of the measures that we studied.

Besides failing to consider every possible admissions variable, we were also unable to

correct for every factor that can artificially depress validity estimates. In particular, we did not

account for the possible effects of restriction in the range of first-year averages (and in

preadmission measures) that can result from the academic dismissal of poor performing students.

The data suggested, however, that the rate of attrition for the first-year veterinary students in the

35

participating schools was relatively low, thereby reducing the need to correct for this factor. This

low rate of attrition has been noted previously by Shane and Talbot (1989), who reported an

incidence of less than 5% on average for U.S. colleges of veterinary medicine, with slightly over

half of the loss coming during the first year. Thus, our failure to consider first-year attrition is

probably not a severe limitation, and again is likely to have resulted in underestimating validity

slightly.

Our estimates may be slightly conservative from another perspective as well, because

statistical corrections were made only on the basis of applicants to each institution, not to the

larger population of GRE test takers. This slight under-correction results from the fact that

veterinary school applicants are already a somewhat self-selected group, as evidenced by GRE

scores that are slightly higher and somewhat less variable than those of the general population of

GRE test takers. Had we corrected back to all veterinary medical applicants before they sorted

themselves to particular schools, the validity estimates might have been somewhat higher.

Finally, one of the strengths of the study � the use of statistical formulae to correct for the

effects of selection � is also a possible limitation. Our best and final estimates of validity are, in

the final analysis, extrapolations to students who applied to, but who did not necessarily enroll in

veterinary school. Although the procedures that we used have stood the test of time, the resulting

estimates are, nonetheless, extrapolations that, in a very real sense, �go beyond the data.�

Further, the accuracy of these extrapolations depends on certain assumptions about the data.

Given the selective nature of veterinary medical school admissions, the corrected estimates are

often significantly higher than those observed for enrolled students. They may therefore be

viewed with suspicion by some skeptics. However, given the impracticality of conducting the

ideal validity study � that is, by admitting all applicants in order to eliminate the effects of

selection � our estimates, though extrapolations, must suffice as the best indications of true

validity that can be mustered.

Conclusion

The major outcome of this study is the evidence it provides of the validity of GRE

General Test scores and undergraduate grades in the context of veterinary school admissions.

36

The study is significant, because it constitutes, we believe, the largest, most systematic study to

date of the validity of GRE scores for one major user of GRE test scores. The effort is also

noteworthy, we believe, because it took advantage of a relatively rare opportunity to apply tried-

and-true validity study techniques � an occasion that seldom presents itself for most validity

studies. We were, therefore, able to address many of the fundamental shortcomings of earlier

validity studies, thus enabling more accurate estimates of the true value of the GRE General Test

than usually possible. Assuredly, this study does not constitute the last word on the validity of

the GRE General Test. However, it does, we feel, add to the accumulation of information about

the utility of the GRE General Test for facilitating admissions decisions in post-baccalaureate

educational settings.

37

References

American Educational Research Association /American Psychological Association /National Council on Measurement in Education. (1999). Standards for educational andpsychological testing. Washington, DC: American Educational Research Association.

Andre, T., & Hegland, S. (1998). Range restriction, outliers, and the use of the GraduateRecord Examination to predict graduate school performance. American Psychologist, 53, 574-575.

Baydar, N., Dugan, M. K., Grady, W. R., & Johnson, T. R. (1995). The GMAT registrantsurvey: Wave II report: An early look at GMAT registrants� progress toward an MBA degree.Seattle, WA: Battelle Memorial Institute�s Center for Public Health Research and Evaluation.

Boldt, R. F. (1986). Generalization of GRE General Test validity across departments(GRE Board Professional Report No. 81-13P; ETS Research Report No. RR-86-46). Princeton,NJ: Educational Testing Service.

Briel, J. B., O�Neill, K. A., & Scheuneman, J. D. (Eds.). (1993). GRE technical manual:Test development, score interpretation, and research for the Graduate Record ExaminationsProgram. Princeton, NJ: Educational Testing Service.

Cohen, J. (1977). Statistical power analysis for the behavioral sciences (Rev. ed.). NewYork: Academic Press.

Confer, A. W. (1990). Preadmission GRE scores and GPAs as predictors of academicperformance in a college of veterinary medicine. Journal of Veterinary Medical Education, 17,56-62.

Confer, A. W., & Lorenz, M. D. (in press). Selectivity of preprofessional institutions andpredictors of first-year academic performance in a veterinary college. Journal of VeterinaryMedical Education.

Confer, A. W., Turnwald, G. H., & Wollenburg, D. E. (1993). Correlation of objectiveand subjective admission criteria with first-year academic performance. Journal of VeterinaryMedical Education, 20, 73-82.

Confer, A. W., Turnwald, G. H., & Wollenburg, D. E. (1995). Influence of selectivity ofpreprofessional institution attended on first-year academic performance in a veterinary college.Journal of Veterinary Medical Education, 22, 8-11.

Cornell, D. G. (1998). Residual validity is not meaningful. American Psychologist, 53,575-576.

Darlington, R. B. (1998). Range restriction and the Graduate Record Examination.American Psychologist, 53, 572-573.

38

Educational Testing Service (1995). Student instructional report II. Princeton, NJ:Author.

Educational Testing Service (1998). 1998-99 GRE guide to the use of scores. Princeton,NJ: Author.

Friedman, B. A., & Niedzwiedz, E. R. (1976). Validation of the Veterinary Aptitude Testat three colleges of veterinary medicine. Journal of Veterinary Medical Education, 3(1), 13-16.

Gulliksen, H. (1950). Theory of mental tests. New York: John Wiley & Sons.

Hambleton, R. K. (1998, August). GRE in admissions decisions: Evidence for validityand some suggestions for the future. Paper presented at the annual meeting of the AmericanPsychological Association, San Francisco, CA.

Hartnett, R. T., & Willingham, W. W. (1980). The criterion problem: What measure ofsuccess in graduate education? Applied Psychological Measurement, 4, 281-291.

Hedges, L. V. (1988). The meta-analysis of test validity studies: Some new approaches.In H. Wainer & H. I. Braun (Eds.), Test validity (pp. 191-212). Hillsdale, NJ: Lawrence Erlbaum.

Held, J. D., & Foley, P. P. (1994). Explanations for accuracy of the general multivariateformulas in correcting for range restriction. Applied Psychological Measurement, 18, 355-367.

Huitema, B. E., & Stein, C. R. (1993). Validity of the GRE without restriction of range.Psychological Reports, 72, 123-127.

Hunter, J. E., & Schmidt, F. L. (1990). Methods of meta-analysis: Correcting error andbias in research findings. Newbury Park, CA: Sage Publications.

Julius, M. F., & Kaiser, H. E. (1978). Preveterinary medical grade point averages aspredictors of academic success in veterinary college. Journal of Veterinary Medical Education,5, 162-164.

Kearney, M. T., Shane, S. M., & Tasker, J. B. (1985). The value of the Medical CollegeAdmissions Test for selection of applicants to a veterinary school. Journal of Veterinary MedicalEducation, 11, 54-62.

Kelman, E. G. (1982). Predicting success in veterinary medical college. Journal ofVeterinary Medical Education, 8, 92-94.

Kuncel, N. R., Campbell, J. P., & Ones, D. S. (1998). Validity of the Graduate RecordExamination: Estimated or tacitly known? American Psychologist, 53, 567-568.

Kuncel, N. R., Hezlett, S. A., & Ones, D. S. (2000). A comprehensive meta-analysis ofthe predictive validity of the Graduate Record Examinations: Implications for graduate studentselection and performance. Psychological Bulletin, 127, 162-181.

39

Latshaw, W. K. (1982). The use of multivariate analysis to analyze the admissionsprocess. Journal of Veterinary Medical Education, 9, 6-9.

Layton, W. L. (1952). Predicting success of students in veterinary medicine. Journal ofApplied Psychology, 36, 312-315.

Linn, R. L., & Dunbar, S. B. (1982). Predictive validity of admissions measures:Corrections for selection on several variables. Journal of College Student Personnel, 23, 222-226.

Linn, R. L., Harnisch, D. L., & Dunbar, S. B. (1981). Corrections for range restriction:An empirical investigation of conditions resulting in conservative corrections. Journal of AppliedPsychology, 66, 655-663.

Linn, R. L., & Hastings, C. N. (1984). A meta-analysis of the validity of predictors ofperformance in law school. Journal of Educational Measurement, 21, 245-259.

Melchert, T. P. (1998). Support for the validity of the Graduate Record Examination.American Psychologist, 53, 573-574.

Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp.13-103). New York: American Council on Education / Macmillan.

Miller, C. E., Barrett, G. V., & Doverspike, D. (1998). A case study versus science.American Psychologist, 53, 569-570.

Muchinsky, P. M. (1996). The correction for attenuation. Educational and PsychologicalMeasurement, 56, 63-75.

Niedzwiedz, E. R., & Friedman, B. A. (1976). A comparative analysis of the validity ofpreadmissions information at four colleges of veterinary medicine. Journal of VeterinaryMedical Education, 3, 32-38.

Noeth, R. J., Smith, D., Stockton, J. J., & Henry, C. A. (1974). Predicting successvariables in the study of veterinary science and medicine. Journal of Educational Research, 67,213-215.

Pearlman, K., Schmidt, F. L., & Hunter, J. E. (1980). Validity generalization results fortests used to predict job proficiency and training success in clerical occupations. Journal ofApplied Psychology, 65, 373-406.

Pearson, K. (1903). Mathematical contributions to the theory of evolution: XI. On theinfluence of natural selection on the variability and correlation of organs. Transactions of theRoyal Society, London, Series A, 200, 1-66.

40

Ramist, L., Lewis, C., & McCamley, L. (1990). Implications of using freshman GPA asthe criterion for the predictive validity of the SAT. In W. W. Willingham, C. Lewis, R. Morgan,& L. Ramist (Eds.), Predicting college grades: An analysis of institutional trends over twodecades (pp. 253-288). Princeton, NJ: Educational Testing Service.

Ramist, L., Lewis, C., & McCamley-Jenkins, L. (1994). Student group differences inpredicting college grades: Sex, language, and ethnic groups. (College Board Report No. 93-1;ETS Research Report No. 94-27). New York: College Entrance Examination Board.

Roznowski, M. (1998). The Graduate Record Examination: Testing. AmericanPsychologist, 53, 570-572.

Ruscio, J. (1998). Gatekeeping, compensation, and fallibility. American Psychologist, 53,568-569.

Schmidt, F. L., Law, K., Hunter, J. E., Rothstein, H. R., Pearlman, K., & McDaniel, M.(1993). Refinements in validity generalization methods: Implications for the situationalspecificity hypothesis. Journal of Applied Psychology, 78, 3-12.

Shane, S. M., & Kearney, M. T. (1989). Evaluation of criteria for admission to a schoolof veterinary medicine. Journal of Veterinary Medical Education, 16, 40-44.

Shane, S. M., & Talbot, R. B. (1989). A review of student attrition in United Statesschools of veterinary medicine. Journal of Veterinary Medical Education, 16, 14-16.

Spearman, C. (1904). The proof and measurement of association between two things.American Journal of Psychology, 15, 72-101.

Sternberg, R. J., & Williams, W. M. (1997). Does the Graduate Record Examinationpredict meaningful success in the graduate training of psychologists? A case study. AmericanPsychologist, 52, 630 � 641.

Thayer, P. W., & Kalat, J. W. (1998). Questionable criteria. American Psychologist, 53,566.

Veterinary Medical College Application Service. (1995). VMCAS application for the1996-97 academic year. Oakland, CA: Author.

Willingham, W. W. (1990). Conclusions and implications. In W. W. Willingham, C.Lewis, R. Morgan, & L. Ramist (Eds.), Predicting college grades: An analysis of institutionaltrends over two decades (pp. 85 � 99). Princeton, NJ: Educational Testing Service.

41

Appendix

Summary of Prior Validity Studies in Veterinary Medical School Admissions

Summary of Prior Validity Studies in Veterinary Medical School Admissions

Study Location Sample size Predictors Criteria Selected findings

Confer(1990)

Oklahoma StateUniversity

619 (ninesuccessiveclasses)

Undergraduate GPA in requiredcourses, cumulativeundergraduate GPA, GREGeneral Test scores

GPA in each offour years ofveterinary school

The most consistently predictive combination ofvariables tended to be undergraduate GPA(either cumulatively or in required courses) andsome GRE General Test scores (usuallyquantitative). Multiple correlations ranged from.19 to .49.

Confer &Lorenz (inpress)


430 (sixsuccessiveclasses)

Type of undergraduateinstitution, cumulativepreveterinary GPA, GPA incourses required for admission,GRE General Test scores, GREBiology Test scores, grades inindividual courses required foradmission

GPA in first-year ofveterinarymedical school

Significant correlations were found betweenfirst-year veterinary GPA and cumulativeundergraduate GPA (r = .46), GRE quantitativescore (r = .41), GRE Biology (r = .40), GPA incourses required for admission (r = .39), GREverbal score (r = .30), and GRE analytical score(r = .29).

Confer,Turnwald, &Wollenburg(1993)


357 (fivesuccessiveclasses)

Cumulative undergraduate GPA,GPA in courses required foradmission, GRE General Testscores, GRE Biology Testscores, interview score, scorebased on information inapplication file

First-year GPAin veterinaryschool

Stepwise regression failed to detect a set ofpreadmission variables that was predictive ofGPA for all classes. Various sets had multiplecorrelations of .51 to .70 with first-year GPA;cumulative undergraduate GPA tended to be themost consistent predictor; GRE scores, filescores, and interview scores also contributed toprediction.

Confer,Turnwald, &Wollenburg(1995)


356 (fivesuccessiveclasses)

Type/selectivity ofundergraduate institutions

Attrition, GPAin first-year ofveterinarycollege

Veterinary college grades and attrition ratesdiffered according to the kind of undergraduateschool attended.

42

Summary of Prior Validity Studies in Veterinary Medical School Admissions (continued)


Friedman &Niedzwiedz(1976)

University ofMinnesota, PurdueUniversity,University ofIllinois

230 first-yearstudents

VCAT scores GPA in first-yearof veterinaryschool

The science subscale of the VCAT was the mosteffective predictor for two of the three schools.

Julius &Kaiser(1978)

Kansas StateUniversity

398 (fivegraduatingclasses)

Undergraduate GPA overall, inlast 45 credit hours, in requiredscience courses, in coursestaken at Kansas State, and incourses required for admissionto the veterinary college

GPA in each offour years ofveterinaryschool,cumulative GPA

GPA in courses required for admissioncorrelated from .49 to .32 with each year's GPAand .47 with cumulative GPA.

Kearney,Shane, &Tasker(1985)

Louisiana StateUniversity

339 (fivegraduatingclasses from1978 through1982)

MCAT scores, GPA in requiredcourses, GPA in sciencecourses, and GPA in last 45credit hours, interviews

GPA in each offour terms,cumulative GPA

MCAT scores were consistently found tocontribute to prediction, as was GPA in requiredcourses and GPA in last 45 credits.

Kelman(1982)

Colorado StateUniversity

128 (oneclass)

Undergraduate GPAs (inrequired and elective coursesand cumulative), GRE GeneralTest scores, interviews,biographical data

GPA in each ofeight terms,cumulative GPAin veterinaryschool

Undergraduate GPA in required courses wasbest predictor (r = .36) of cumulative GPA;GRE verbal score and GRE quantitative scorecorrelated slightly (r = .21, .16) with GPA.

Latshaw(1982)

University ofSaskatchewan

4 veterinaryschoolclasses (nsnot specified)

Undergraduate GPA in earliestand latest preprofessional year,GRE General Test scores,interviews, references

GPA in each offour years ofveterinary school

GPA in latest preprofessional years was mostpredictive of veterinary school grades in eachyear (r = .34 to .65). GRE verbal scores werepredictive in the first year only (r = .37).

Layton(1952)

University ofMinnesota

87 (twosuccessiveclasses)

Iowa Veterinary Aptitude Testscores, Professional AptitudeTest scores, Strong VocationalInterest Blank (SVIB),preveterinary college grades

GPA in first yearof veterinarymedicine

Variables that contributed significantly toprediction were grades in preveterinary collegecourses, the veterinary scale on the SVIB, andthe preveterinary achievement score of the Iowatest.

43

Summary of Prior Validity Studies in Veterinary Medical School Admissions (continued)


Niedzwiedz &Friedman(1976)

Ohio StateUniversity,University ofMinnesota,PurdueUniversity,University ofIllinois

365 (freshmanclasses at fouruniversities)

Biographical information,undergraduate grades,interview ratings, letters ofrecommendation, VeterinaryAptitude Test (VAT) scores,admission committee ratings

GPA in first year ofveterinary school

Undergraduate grades in particular coursesrelated to first-year GPA at three schools. VATscores were related to first-year performance attwo schools. Other predictors were generally notrelated to first-year performance.

Noeth, Smith,Stockton, &Henry (1974)

PurdueUniversity

100 (twosuccessiveclasses)

VAT subscores, GPA for allpreveterinary college courses,GPA in science courses, highschool rank, SAT scores

GPA in first-semester ofveterinary school

The single best predictor was preveterinarycollege GPA (r = .58). The addition of foursubscores from the VAT increased the multiplecorrelation to .67.

Shane &Kearney(1989)

Louisiana StateUniversity

389 (fivegraduatingclasses from1983 through1987)

MCAT scores, cumulativepreprofessional GPA, GPA inrequired courses, GPA in last45 credit hours

GPA in each of sixsemesters ofveterinary school,cumulative GPA

The most consistent sets of predictors wereGPA in the last 45 credit hours and GPA inrequired courses; MCAT scores contributed toprediction only in the first year.

44

54020-008638 • Y61P.4 • Printed in U.S.A.I.N. 991743

®

powers gre 98-09r ets-rr-01-10 final

Documents