predicting college grades: the value of achievement goals in supplementing ability measures

Assessment in EducationVol. 14, No. 2, July 2007, pp. 233–249

ISSN 0969-594X (print)/ISSN 1465-329X (online)/07/020233–17© 2007 Taylor & FrancisDOI: 10.1080/09695940701479709

Predicting college grades: the value of achievement goals in supplementing ability measuresJohn W. Young*Rutgers University, USATaylor and FrancisCAIE_A_247847.sgm10.1080/09695940701479709Assessment in Education0969-594X (print)/1465-329X (online)Original Article2007Taylor & Francis142000000July [email protected]

Achievement goal theory is an important theoretical framework for understanding achievementmotivation. In previous studies, a mastery orientation has been shown to be related to students’interest, while a performance orientation has been found to be predictive of academic performanceoutcomes such as course grades. In this study, the two mastery sub-scores from the Multiple GoalsTheory Measure (MGTM) of academic motivation, which was developed specifically from achieve-ment goal theory, was found to be predictive of college grades for a sample of 257 undergraduatesat a public university in the north-eastern United States. Additionally, the results support a trichot-omous model of achievement orientations comprising mastery approach, performance approach,and avoidance. The MGTM appears to hold promise as a diagnostic tool, but additional researchis required on its resistance to faking and other threats to validity.

Achievement goal theory has become an influential social cognitive theory of achieve-ment motivation over the past two decades (Nichols, 1984; Dweck, 1986; Ames &Archer, 1988; Dweck & Leggett, 1988; Ames, 1992; Elliot & Harackiewicz, 1996;Elliot & Church, 1997; Midgley et al., 1998; Elliot, 1999; Pintrich, 2000a,b).Achievement goals represent the motivation behind achievement behaviours in aparticular setting (Nichols, 1984). The theory postulates that there are two differenttypes of achievement goal orientations, mastery and performance, with two distinctvalences for each orientation, approach and avoidance. Mastery goals reflect astudent’s intent to develop competence by acquiring new knowledge or skills in alearning activity. Performance goals reflect a student’s desire to demonstrate compe-tence as compared to other students. An orientation to competence can be eitherpositive (as in an approach orientation) or negative (as in an avoidance orientation),which may depend on the learning activity or situation. This crossing of goals with

*Educational Testing Service, Center for Validity Research, Mail Stop 10–R, Princeton, NJ 08541,USA. Email: [email protected]

234 J. W. Young

orientations results in a total of four goal orientations: mastery approach, perfor-mance approach, mastery avoidance, and performance avoidance.

A person who adheres to a mastery approach goal orientation is focused on goalssuch as learning as much as possible, increasing his or her competence, or striving tomaster challenging material. An individual who is mastery avoidance oriented ischaracterized as one who seeks to avoid a lack of mastery or a failure to learn. Aperformance approach oriented person attempts to demonstrate his or her abilities inrelation to others or is interested in proving their self-worth publicly. Although aperson who is performance approach oriented defines success primarily by means ofcomparison, this type of goal orientation is not necessarily a negative one. In fact, anumber of theorists have remarked that having both mastery and performance goalsare healthy, adaptive, and can lead to elevated levels of motivation and success.Lastly, a person who subscribes to a performance avoidance orientation is focused onavoiding the appearance of incompetence or of lacking in ability, especially relative totheir peers (Wolters, 2004).

Elliot and colleagues (Elliot & Harackiewicz, 1996; Elliot & Church, 1997; Elliot,1999) proposed a revised conceptualization of achievement motivation goals, atrichotomous goal framework with three independent achievement goals: masteryapproach, performance approach, and an omnibus avoidance goal. Mastery avoid-ance goals, such as the avoidance of misunderstanding material or of not masteringor successfully completing a task, are not included in the trichotomous achievementframework. Elliot and his colleagues have also suggested that it may not be productiveto view all performance goals as being maladaptive or in opposition to mastery goals.In their research, Elliot and his colleagues showed that mastery approach goals arerelated to interest, performance approach goals are positively related to actual perfor-mance, and performance avoidance goals are related to undesirable outcomes such aslow interest and weak performance.

Because it is possible to pursue simultaneously more than one achievement goal,researchers have examined the independent and interactive effects of these multiplegoals (Barron & Harackiewicz, 2000; Linnenbrink & Pintrich, 2000; Harackiewicz etal., 2002b; Pintrich, 2000b). In his review, Pintrich (2000c) points out that the fourdifferent orientations might be causally linked in substantially different ways tooutcomes such as attributions, efficacy, affect, self-regulation, persistence, andchoice. For example, mastery approach goals may be positively related to a variety ofadaptive outcomes, but some adaptive outcomes may also be linked to performanceapproach goals. Pintrich also indicated that the mastery avoidance goal orientation is,at present, somewhat undefined theoretically and operationally in the research litera-ture on achievement goals. However, Elliot & McGregor (2001) found empiricalsupport for the existence of mastery avoidance goals, which were likely to be adoptedby students who harbour fear of failure, which is activated in challenging classroomsituations.

Prior research on achievement goal theory indicates that mastery goals are predic-tive of students’ interest, while performance goals can predict students’ achievementas measured by course grades and grade point averages (GPAs) (Harackiewicz et al.,

Predicting college grades 235

2002b). A review by Harackiewicz et al. (2002a) summarized the results of numerousstudies of goal effects on performance and interest outcomes. In general, performanceapproach goals were low to moderately correlated with academic performanceoutcomes such as exam scores, course grades, and semester and subsequent GPAs.Mastery goals tended to have little to no correlation with any performance outcomes.In contrast, mastery goals were generally moderately correlated with interestoutcomes such as interest in class, enjoyment of lectures, and measures of continuedinterest. A few studies showed a weak association between performance approachgoals and interest outcomes, while most studies showed no correlation.

Two studies by Harackiewicz and her colleagues (Harackiewicz et al., 2000; Harac-kiewicz et al., 2002b) of students enrolled in introductory psychology classes foundthat mastery goals predicted later course interest, but not course grades. In contrast,performance goals were found to predict grades, but not interest. Harackiewicz et al.(2000) found that mastery goals predicted enrolment in later psychology courses (r =.18). Performance goals predicted two measures of academic performance in thesubsequent three semesters after the initial data collection, overall grades (r = .12)and grades in psychology courses (r = .21). In a further follow up, seven years afterthe initial study, Harackiewicz et al. (2002b) found that mastery goals predictedcontinued interest, while performance goals predicted two measures of long-termacademic performance, cumulative GPA and GPA in all psychology courses. Harac-kiewicz et al. (2002b) argued that their findings support a multiple goals perspective.

In Urdan’s (1997) review, he reported that research on achievement goals hasgenerally been carried out using two methods: experimental manipulations andsurveys, with the latter design being favoured in more recent studies. For example,Midgley et al. (1998) developed an 18-item instrument that assessed three achieve-ment goal orientations, which they labelled task orientation, performance approachorientation, and performance avoidance orientation. This measure has been used ina number of studies with samples of elementary and middle school students.However, the use of surveys in field research for the ‘measurement of goals hascontinued to be an imprecise and varied endeavor’ (Urdan, 1997, p. 106). In partic-ular, Urdan stated that the varied and imprecise measurement of goals, especiallyrelative ability (or performance) goals, has produced an unclear pattern of resultsregarding the effects of pursuing relative ability goals. Thus, our understanding ofachievement goal theory may be hampered by a lack of theoretically based and scien-tifically sound instruments for accurately measuring an individual’s goal orientations.

The Multiple Goals Theory Measure of academic motivation

In designing the Multiple Goals Theory Measure (MGTM) of academic motivation,we adopted Pintrich’s (2000a) perspective that achievement motivation goals are, bydefinition, cognitive, that they can be brought into conscious awareness, and thatindividuals can access them. Therefore, the main psychometric challenge is to designmeasures that provide valid and reliable measures of these goals. This includes‘measuring the appropriate level of a goal such as a very specific target goal for a

236 J. W. Young

particular situation or general goal orientations toward school work as well asappropriate domain specificity (e.g., math, science, reading, etc.)’ (Pintrich, 2000a,p. 97). Thus, one of our major aims in developing the MGTM was to measure bothmastery and performance goals more accurately than is possible using currentlyavailable instruments for assessing achievement goals.

Most existing scales that assess achievement motivation are based on self-reportedinformation using relatively simple items with one of the following response formats:Likert response formats, rank order formats, and open-ended formats. One of themajor goals in developing the MGTM was to reduce the social desirability inresponding and/or faking that are inherent problems in measures using self-reportedinformation. Current discussions of these problems can be found in Heggestad et al.(2006) and Kyllonen et al. (2005). To deal with these concerns, we investigated theuse of vignettes to assess achievement motivation. We found that most existingmeasures that incorporated vignettes are currently used to measure constructs otherthan motivation, or are being used to measure motivation in non-educationalcontexts (e.g., in business settings). In addition, vignettes have been used to createscales for measuring tacit knowledge and practical intelligence (Wagner & Sternberg,1986). The use of vignettes to measure students’ motivation represents a novelapproach to developing assessments, and this study may provide important insightsinto the benefits of including vignettes in assessment instruments.

As an elicitation tool in qualitative and quantitative research, vignettes are shortscenarios or stories in written or pictorial form which participants can respond to orcomment upon. Vignettes have been used in psychological research since the 1950s(Anderson & Anderson, 1951), and have been used in social psychological and socio-logical research to study attitudes, perceptions, beliefs, and norms (Finch, 1987). Insocial research, participants are asked to respond to a particular situation by statingwhat they would do or how they imagine that a third person would react. Whencompared with items typically used in self-reported measures, vignettes may have anadvantage in addressing social desirability issues because of the distance created by ahypothetical situation from a participant’s direct experience (Gould, 1996). The useof vignettes in the study of sensitive topics is well documented (Finch, 1987; Hughes,1998; Barter & Renold, 2000; Hughes & Huby, 2002). Vignettes are seen as usefulin ‘the study of potentially difficult topics of inquiry as they can help desensitizeaspects of these for participants’ (Hughes & Huby, 2002, p. 384). Gould’s (1996)study of nurses’ infection control practices found that vignettes reduced the impactof social desirability factors on participants’ responses. Gould argues that the use ofvignettes may help in avoiding Hawthorne and observer effects, and is one of the mainrationales for using them. Additionally, some studies have found that responses tovignettes reflect how individuals actually respond in reality (Rahman, 1996; Hughes& Huby, 2002).

The measure of achievement goals that was developed was named the MultipleGoals Theory Measure of academic motivation so as to reflect a specific applicationof achievement goal theory for identifying motivational goal orientation in academicactivities at the high school level. The MGTM was developed in consultation with the


late Paul Pintrich, one of the major researchers on achievement goal theory. TheMGTM consists of 25 vignettes, each a one-paragraph hypothetical situation involv-ing a school-based assignment or activity, along with 4 Likert-type response itemswith one item keyed to one of the four goal orientations in the multiple goals perspec-tive on achievement goal theory (mastery approach, mastery avoidance, performanceapproach, or performance avoidance). The vignettes describe school-related tasksand activities, including exams, projects, and extra credit assignments, in one of fourcommon high school subjects: English, history, mathematics, and science. Althoughdifferent subject areas were used in the vignettes, we believed that goal orientationsare broader academic orientations that are domain-general rather than domain-specific. Responses to the items are given on a 5-point Likert scale ranging from ‘NotLikely’ to ‘Very Likely’. The items for the different vignettes were randomly orderedso as to avoid response bias resulting from order effects. Instructions for completingthe MGTM and a sample vignette are included as Appendix A. The first responseitem associated with this vignette measures mastery approach; the second item,performance approach; the third item, mastery avoidance; and the fourth item,performance avoidance. In contrast to other existing measures of motivation that relyprimarily or exclusively on respondents’ self-reports, the MGTM was developed withthe intent of minimizing socially desirable responding and/or faking.

The MGTM does not measure cognitive abilities, as evidenced by its low corre-lations with SAT scores, high school grades, and total score on seven multiplechoice items associated with a disclosed Graduate Record Examination (GRE)reading comprehension passage. The internal consistency reliability, based onCronbach’s coefficient alpha, for each of the four MGTM sub-scales is acceptable.Each sub-scale is composed of 25 items, with one item from each vignette; thecoefficient alpha value for the scales ranges from .85 to .93. In order to testwhether using different academic subjects in the vignettes affected participants’responses, we grouped the vignettes by subject areas and correlated the sub-scoreswithin each goal orientation. Given the high internal consistency of the MGTMsub-scores, it was not surprising to find that the correlations across subject areaswere very high, with most correlations above .80. This supports our belief that thegoal orientations, as measured by the MGTM, are domain-general rather thandomain-specific, and is consistent with Dweck & Leggett’s (1988) view that goalsrepresent a somewhat stable orientation across situational contexts. Additionalinformation about the instrument development and validation processes for theMGTM can be found in Young & Robustelli (2004).

Overview of the present study

In this study, the MGTM was used as a supplement to the ability measures of SATscores and high school grades in order to predict college grades. A rationale for creat-ing the MGTM is to determine if information on students’ achievement motivationcan improve the prediction of their academic performance. As achievement goaltheory has not previously been used as an individual diagnostic tool for students, this

238 J. W. Young

study represents an innovative application of the theory. For this study, we wereinterested in answering the following specific research questions:

● Are any of the MGTM sub-scores significantly correlated with college grades?● Are mastery or performance sub-scores more highly correlated with college grades?● Are any of the MGTM sub-scores a significant incremental predictor of college

grades, beyond what can be predicted from SAT scores and high school grades?● Does use of MGTM sub-scores improve the predictive validity of college grades

for all students, particularly for male students and racial/ethnic minority students?

Previous studies have found that performance goals, but not mastery goals, arepredictive of students’ grades (see review by Harackiewicz et al., 2002a). We wantedto determine whether the use of a new measure that was developed from achievementgoal theory would yield similar results or not. In addition, we were particularly inter-ested in the usefulness of the MGTM as a supplemental tool to the traditionalacademic college admissions measures for predicting college grades. The specificinterest on the prediction of college grades for male students and minority studentsstems from the literature of predictive validity studies of the SAT, which documentsthat, for these two demographic groups, the correlation of SAT scores with collegegrades is significantly lower than for female students and white students (see reviewby Young, 2001). If SAT scores are not as informative about later academic perfor-mance for some groups of students, then having measures available that are predictiveof their performance would be especially useful to admissions officers at selectivecolleges and universities and for identifying students who may be potentially at risk ofacademic difficulties.

Method

Participants

The sample for the study consisted of 257 undergraduate students at a researchuniversity in the north-eastern United States who were enrolled in either Introductionto Psychology or Introduction to Educational Psychology during the spring semesterof 2003. These students are typically in their first or third year of study at the univer-sity, as Introduction to Psychology is a popular course among first-year students whileIntroduction to Educational Psychology serves as an entry course for the teachereducation programmes. Since enrolment in one of these two courses was required inorder to participate in this study, the resulting sample of subjects is not representativeof all students at this university. Specifically, the sample has an under-representationof students majoring in the physical sciences and engineering fields. For most of thevariables based on data from the university registrar, there is missing information onapproximately 4% of the students.

In the sample, 171 (69%) of the students are female and 76 (31%) are male. All,except one student, are US citizens. With regard to racial/ethnic identity, 178 (72%)students are white, 31 (13%) are Asian–American, 13 (5%) are Hispanic, 9 (4%) are


African–American, 9 (4%) are other, and 7 (3%) are Puerto Rican. With regard tocollege enrolment, 75 (31%) of the students were in their first year of study, 38 (16%)were in their second year, 71 (29%) were in their third year, and 59 (24%) were intheir fourth year. The five most common majors were undeclared (110 students, or43%), psychology (25 or 10%), history (24 or 9%), English (12 or 5%), and pre-business (11 or 4%). The majority of students were enrolled in one of the liberal artscolleges (177 students or 72%), with another 32 students (13%) enrolled in theschool of education.

Measures

The participants in the study completed the MGTM as well as seven multiple-choiceitems associated with a disclosed Graduate Record Examination (GRE) readingcomprehension passage. Total score from the reading comprehension items was usedto assess the construct validity of the MGTM, since we wanted to ascertain that theMGTM sub-scores did not function simply as measures of ability. Admissions,demographic, and university grade information for the subjects was obtained from theuniversity’s registrar’s office and merged with the MGTM responses. This informa-tion included the sex, race, and citizenship of the student, as well as SAT Verbal andMathematical scores (SATV and SATM), high school rank in class percentile (HSR),university current year and cumulative GPAs, major field of study, and number ofdegree credits completed. Because students in the study were at different points intheir college academic career, it was decided that current year GPA would be a morevalid criterion than first-year GPA since it is contemporaneous with participation inthis study for all subjects. Note that this university uses SAT scores with the trailingzero deleted so that the range of scores is from 20 to 80. HSR is computed from astudent’s rank in class divided by the size of the high school class, which is subtractedfrom one and then multiplied by 100. HSR ranges from a low of 0.0 to a high of100.0. This university does not use any version of a high school GPA in makingadmissions decisions.

Statistical analyses

Descriptive summary statistics and correlation coefficients for all of the variablesincluded in the study were computed. In addition, Cronbach’s coefficient alpha wascalculated to determine the internal consistency reliability of the four MGTM sub-scales, and to identify potentially problematic items. Multiple regression analyseswere conducted to determine the predictive validity of the traditional admissionsmeasures (SAT scores and HSR), as well as the incremental validity of the MGTMsub-scores in forecasting college GPA. Correlation and regression analyses wereconducted on the entire sample of participants. In addition, because sex and racedifferences in predicting college grades have previously been found (Young, 2001),regression analyses that included indicator variables for these demographic character-istics were also conducted.

240 J. W. Young

Results

The results of the study are presented in the following sections: Descriptive Statistics,Correlations, Regression Analyses, and Sub-group Analyses.

Descriptive statistics

Summary statistics for all of the key variables included in the study are shown inTable 1.

Correlations

Correlation coefficients were computed among: the three academic admissions vari-ables; the two university GPAs, yearly and cumulative; scores on GRE items; and thefour MGTM sub-scale scores. The matrix of these correlation values is shown inTable 2. The correlations of total score on the GRE reading comprehension items withthe MGTM sub-scale scores were used to assess the construct validity of the MGTM.The GRE total score is significantly correlated with all three admissions variables andwith both of the GPAs. As expected, GRE total score is not significantly correlatedwith any of the MGTM sub-scores. Because GRE total score is not correlated withany of the MGTM sub-scores, this provides evidence that the MGTM sub-scales arenot measures of ability. The MGTM sub-scales appear to measure characteristicsdistinct from those assessed by the GRE items. Two of the MGTM sub-scores,mastery approach and performance approach, are uncorrelated with any of theacademic admissions variables. This provides further evidence that these sub-scoresare measuring traits different from those assessed by SAT scores and high schoolgrades. The other two MGTM sub-scores, mastery avoidance and performanceavoidance, showed negative correlations with the academic admissions variables.

Correlations among the four MGTM sub-scores, reported in Table 2, showedvarying degrees of association among the scores. In fact, the matrix of correlations has

Table 1. Descriptive summary statistics for selected study variables

Variable N Mean Std. Dev. Minimum Maximum

SATV 213 571.08 77.41 330 800SATM 213 589.91 78.38 340 800HSR 175 80.57 14.77 36 100Year GPA 238 3.15 0.70 0.33 4.00Cum. GPA 243 3.10 0.66 0.33 4.00Degree credits 243 60.87 33.19 6.00 126.00Mastery approach 253 86.85 17.64 39 125Mastery avoidance 253 59.60 12.88 25 100Performance approach 253 59.60 17.25 25 114Performance avoidance 252 45.40 15.75 25 111


values similar to those reported by Elliot & McGregor (2001) with three notableexceptions. First, we reported a correlation of -.35 for mastery approach with masteryavoidance, while Elliot & McGregor reported a value of .35. Given that the twomastery goals have opposing valences, it seems reasonable to expect that these twosub-scores would have a negative association rather than a positive one. Second, wefound a stronger negative correlation between mastery approach and performanceavoidance (-.37) than did Elliot & McGregor (-.05). Third, we found a much stron-ger positive correlation between the two avoidance orientations (.79) than for Elliot& McGregor (.36). The results from this study indicated that mastery avoidance doesnot appear to be a construct distinctly different from performance avoidance (asshown by the high correlation of .79 between these two MGTM sub-scales), a findingthat indirectly supports the three-goal model as specified by Elliot and colleagues(Elliot & Harackiewicz, 1996; Elliot & Church, 1997; Elliot, 1999), where an omni-bus avoidance construct encompasses both mastery and performance orientations.This finding is in contrast to that reported by Elliot & McGregor (2001) who foundthat a model with four goal orientations provided a better fit to their data than dideither of two trichotomous goal models.

Regression analyses

Hierarchical multiple regression analyses were conducted on the sample of partici-pants to determine whether any of the MGTM sub-scores provided incrementalvalidity in predicting college grades beyond what is known from SAT scores and HSgrades. The three traditional admissions variables and the four MGTM sub-scoreswere used to predict each of the two GPAs. In order to model a realistic admissionsscenario, SAT scores and HSR were entered first into the regression equation,followed in stepwise fashion by the four MGTM sub-scores. Since SAT scores andHSR are already being used to make admissions decisions at this university, we are

Table 2. Matrix of correlation coefficients for selected study variables

1 2 3 4 5 6 7 8 9 10

1. SATV2. SATM .49**3. HSR .28** .29**4. Year GPA .20** .16** .17**5. Cum. GPA .26** .23** .18** .92**6. GRE .48** .29** .15* .23** .23**7. Mast. approach .05 .02 .10 .16** .15** .028. Mast. avoidance −.18** −.13* −.15** −.16** −.21** −.11 −.35**9. Perf. approach −.04 .04 .06 .01 −.09 .03 .22** .19**10. Perf. avoidance −.13* −.13* −.06 −.10 −.15** −.06 −.37** .79** .26**

Note: * = p < .05, ** = p < .01.

242 J. W. Young

interested in testing whether any of the MGTM sub-scores added significantly to theprediction of college grades. As noted earlier, each of the three admissions variablesis significantly correlated with both of the GPAs. Because a large proportion ofstudents were missing SAT scores (17%) and/or HSR (36%), we imputed the meansfor these variables in place of missing values in order to maintain the largest samplepossible.

For yearly GPA, none of the three traditional admissions measures were significantpredictors when all three measures were included in the prediction equation. This islikely due to the fact that since these variables are moderately intercorrelated (from.28 to .49), a small degree of multicollinearity affected the results. When we usedSAT total score in place of SATV and SATM, SAT total score was found to be asignificant predictor of yearly GPA (r = .21, p < .01). Mastery approach was a signif-icant predictor of yearly GPA (r = .16, p = .025), and significantly improved theprediction of yearly GPA by increasing the multiple correlation from .21 to .25(results shown in Table 3). For cumulative GPA, SATV was the only one of the threetraditional admissions variables that was a significant predictor. Furthermore,mastery avoidance was a significant predictor of cumulative GPA (r = -.21, p < .01),and significantly improved the prediction of cumulative GPA by increasing themultiple correlation from .27 to .32 (results shown in Table 3). None of the otherMGTM sub-scores significantly increased the prediction accuracy for either of thecollege GPAs.

Sub-group analyses

Analyses of the correlations and hierarchical regression analyses were carried out forsub-groups of students identified by race and sex in order to examine group differ-ences. For male students, three of the MGTM sub-scores, mastery approach, masteryavoidance, and performance avoidance, were more strongly correlated with both ofthe GPAs than any of the admissions variables. Mastery approach was correlated .30with current year GPA and .27 with cumulative GPA; Mastery avoidance was corre-lated −.29 with current year GPA and −.26 with cumulative GPA; Performanceavoidance was correlated −.29 with both of the college GPAs (all correlations aresignificant at p < .05). SATV, which had the highest correlations among the admis-sions variables, was correlated .25 and .26 with the two GPAs, respectively. Forfemale students, only mastery avoidance was significantly correlated with cumulativeGPA (r = −.20, p < .01), while none of the MGTM sub-scores were significantlycorrelated with yearly GPA. All three of the academic admissions variables had equalor higher correlations with the two college GPAs, ranging from .20 to .37.

For minority students (which included a total of 60 African–American, Asian–American, Hispanic, and Puerto Rican students), mastery approach was bettercorrelated with yearly GPA than any of the traditional admissions variables, with acorrelation of .27 (p < .05), while mastery avoidance was better correlated withcumulative GPA than any of the traditional admissions variables, with a correlationof −.27 (p < .05). Lastly, for white students, only mastery avoidance was signifi-


cantly correlated with either GPA, with a correlation of -.16 with current year GPA(p < .05) and −.19 with cumulative GPA (p < .01). All three of the academicadmissions variables had somewhat higher correlations with the two college GPAs,ranging from .16 to .29. The correlation results for the two mastery sub-scores formale students and minority students are not unexpected. It has been well docu-mented that, for these two demographic groups, the correlations of traditionaladmissions measures with college GPA are generally lower than for female students

Table 3. Results from hierarchical multiple regression analyses

Summary of Hierarchical Multiple Regression Analysis for Variables Predicting Current Year GPA (N =257)

Variable B s.e.(B) Beta t-statistic sig. (p<)

Step 1(intercept) 1.734 0.249SATV 0.011 0.007 0.120 1.685 .093SATM 0.005 .0007 0,054 0.752 .452HSR 0.006 0.004 0.0105 1.635 .103

Step 2(intercept) 1.338 0.460SATV 0.011 0.007 0.116 1.643 .102SATM 0.005 0.007 0.056 0.784 .434HSR 0.005 0.003 0.093 1.463 .145Mastery approach 0.005 0.002 0.138 2.248 .025

Note: Multiple R = .208 for step 1, Multiple R = .249 for step 2

Summary of Hierarchical Multiple Regression Analysis for Variables Predicting Cumulative GPA (N =257)


Step 1(intercept) 1.275 0.410SATV 0.015 0.006 0.161 2.296 .023SATM 0.010 0.006 0.108 1.541 .125HSR 0.005 0.003 0.089 1.420 .157

Step 2(intercept) 1.988 0.479SATV 0.013 0.006 0.141 2.025 .044SATM 0.009 0.006 0.103 1.484 .139HSR 0.004 0.003 0.074 1.190 .235Mastery avoidance −0.009 0.003 −0.169 −2.776 .006

Note: Multiple R = .272 for step 1, Multiple R = .319 for step 2.

244 J. W. Young

and white students (Young, 2001). Because the two mastery sub-scores wereshown to be correlated with the college GPAs for males and minority students, theuse of these scores can lead to improved predictive validity of college grades forthese sub-groups.

Additional hierarchical multiple regression analyses were conducted thatincluded two indicator variables, one each for the sex and race of the student.These analyses were carried out to determine if any of the MGTM sub-scoreswould still have predictive power once sex and race differences in the college GPAswere taken into account. The indicator variable for sex was coded 1 for males and2 for females, while the indicator variable for race was coded 0 for white studentsand 1 for minority students. A regression model with an additional term that repre-sented the interaction of sex and race was also tested, but the interaction term wasnot significant in predicting either of the college GPAs. The results from theseanalyses (shown in Table 4) indicated that mastery approach is a significant predic-tor for yearly GPA (p < .05) given the three traditional admissions variables andthe indicator variables for sex and race, and that mastery avoidance is a significantpredictor for cumulative GPA (p < .01) given the presence of the other variablesdescribed above.

Discussion and conclusion

The primary objective of this study was to determine whether use of the MGTMmight lead to improved prediction of college grades for a sample of students at apublic university in the north-eastern United States. The results demonstrated thatthe MGTM mastery sub-scores yielded improved predictive validity, and had partic-ularly high correlations with the two university GPAs for male students and minoritystudents. For these two groups, the traditional admissions measures of SAT scoresand high school grades have not been as effective historically in forecasting academicachievement in college, since the correlations are lower, as is true for females andnon-minority students (Young, 2001). The use of a measure based on achievementgoal theory led to significantly better grade prediction than could be accomplishedwhen only ability measures are used. In contrast to earlier studies, mastery goals, butnot performance goals, were significantly correlated with college grades and were asignificant increment to the ability measures in predicting college grades. This resultis consistent with the explication of achievement goal theory as specified by Harack-iewicz et al. (2000), since mastery goals may be more advantageous in advancedcollege courses which can require greater efforts, deeper understanding, and moresustained interest in a field of study. A likely explanation as to why mastery goals werefound to be predictive of grades in this study is that substantial effort was put forth indeveloping and refining the MGTM, so as to assure fidelity with achievement goaltheory. The MGTM is a significantly longer and more complex instrument than theitems and scales used to measure mastery and performance goals by Harackiewiczand colleagues in their studies (Harackiewicz et al., 2000, 2002a,b). It may well bethat mastery goals are actually predictive of long-term academic performance, as


Table 4. Results from hierarchical multiple regression analyses with group membership variables

Summary of Hierarchical Multiple Regression Analysis for Variables Predicting Current Year GPA (N =257)


Step 1(intercept) 1.458 0.484SATV 0.008 0.007 0.085 1.195 .233SATM 0.009 0.007 0.094 1.274 .204HSR 0.007 0.004 0.121 1.886 .060Gender 0.148 0.095 O.100 1.563 .119Minority −0.299 0.098 −0.188 −3.054 .002

Step 2(intercept) 1.134 0.502SATV 0.008 0.007 0.080 1.146 .253SATM 0.008 0.007 0.090 1.233 .219HSR 0.006 0.003 0.112 1.767 .078Gender 0.125 0.094 0.084 1.318 .189Minority −0.307 0.097 −0.193 −3.150 .002Mastery approach 0.005 0.002 0.134 2.212 .028

Note: Multiple R = .298 for step 1, Multiple R = .326 for step 2.

Summary of hierarchical multiple regression analysis for variables predicting cumulative GPA (N =257)


Step 1(intercept) 0.858 0.460SATV 0.011 0.006 0.123 1.781 .076SATM 0.015 0.007 0.163 2.276 .024HSR 0.005 0.003 0.101 1.618 .107Gender 0.205 0.090 0.142 2.278 .024Minority −0.303 0.093 −0.196 −3.252 .001

Step 2(intercept) 1.572 0.513SATV 0.009 0.006 0.102 1.492 .137SATM 0.015 0.006 0.160 2.260 .025HSR 0.005 0.003 0.085 1.372 .171Gender 0.213 0.089 0.148 2.401 .017Minority −0.303 0.092 −0.196 −3.306 .001Mastery avoidance −0.009 0.003 −0.174 −2.948 .004

Note: Multiple R =.365 for step 1, Multiple R = .403 for step 2

246 J. W. Young

specified in achievement goal theory, but that the use of previously developedinstruments did not successfully capture the different goal orientations. The use ofthe MGTM provides empirical evidence that mastery goals do effectively forecastacademic performance outcomes. In addition, the results from this study providesupporting evidence for a trichotomous framework of achievement goals comprisingmastery approach, performance approach, and an omnibus avoidance goal (Elliot,1999).

Our findings are consistent with the results from the last of five studies conductedby Grant & Dweck (2003). In this study, Grant & Dweck reported that mastery goalswere predictive of final grades earned in General Chemistry. They explained thatbecause the goal items they used occurred in a ‘real world’ setting, and because thecourse required sustained effort by participants over the entire semester, learning (ormastery) goals had a more facilitative effect than performance goals on motivationand performance. They concluded that for courses that involve sustained challenge,mastery goals do positively affect course performance. Additional analysis by Grant& Dweck indicated that the association between mastery goals and course grades aremediated by students’ tendency to engage in deeper processing of the course material.Harackiewicz et al. (2000) and Harackiewicz et al. (2002b) have hypothesized thatmastery goals might prove more advantageous in advanced courses that requiredeeper processing and sustained effort, and thus may predict performance in upperdivision college courses. However, the results from our study do not support thishypothesis as we found that the correlations between mastery approach and bothcollege GPAs were highest for first-year students, not for students with moreadvanced standing at the university.

In conclusion, this initial study of the MGTM, which was developed explicitlyfrom achievement goal theory, indicates that it is a psychometrically sound andpromising measure of achievement motivation. At present, the MGTM appears tobe promising as a diagnostic tool to identify those students entering higher educa-tion who may be at risk for academic difficulties. Individuals with motivationalprofiles that include low mastery approach and high mastery avoidance scoreswould appear to be those most in need of early intervention (such as advising,tutoring, and/or mentoring) to avoid academic failure or dropping out. For high-stakes applications, such as would be the case for use in selective admissionsdecisions, it would be necessary to firmly establish that the MGTM is resistant tofaking and coaching strategies and can maintain its validity as a measure ofstudents’ achievement motivation.

Acknowledgements

This study was supported by a research and development grant from the CollegeBoard of New York, USA. The author is no longer affiliated with Rutgers Universityand is now a Senior Research Scientist at the Educational Testing Service. Theauthor would like to thank the executive editor and the two anonymous reviewers fortheir helpful suggestions.


Notes on contributor

The author is no longer affiliated with Rutgers University and is now a senior researchscientist in the Center for Validity Research at the Educational Testing Servicein Princeton, New Jersey, USA. His Ph.D. is in educational measurement fromStanford University and his research interests are in test validity and fairnessissues. In 1999, he received the Early Career Contribution Award from theAmerican Educational Research Association’s Committee on Scholars of Colorin Education.

References

Ames, C. (1992) Classrooms: goals, structures, and student motivation, Journal of EducationalPsychology, 84(3), 261–271.

Ames, C. & Archer, J. (1988) Achievement goals in the classroom: student learning strategies andmotivation processes, Journal of Educational Psychology, 80(3), 260–267.

Anderson, H. H. & Anderson, G. L. (1951) An introduction to projective techniques and other devicesfor understanding human behaviour (Englewood Cliffs, NJ, Prentice Hall).

Barron, K. E. & Harackiewicz, J. M. (2000) Achievement goals and optimal motivation: a multiplegoals approach, in: C. Sansone & J. M. Harackiewicz (Eds) Intrinsic and extrinsic motivation: thesearch for optimal motivation and performance (New York, Academic Press), 229–254.

Barter, C. & Renold, E. (2000) ‘I want to tell you a story’: exploring the application of vignettes inqualitative research with children and young people, International Journal of Social ResearchMethodology, 3(4), 307–323.

Dweck, C. S. (1986) Motivational processes affecting learning, American Psychologist, 41(10),1040–1048.

Dweck, C. S. & Leggett, E. (1988) A social cognitive approach to motivation and personality,Psychological Review, 95(2), 256–273.

Elliot, A. J. (1999) Approach and avoidance motivation and achievement goals, EducationalPsychologist, 34(3), 169–189.

Elliot, A. J. & Church, M. A. (1997) A hierarchical model of approach and avoidance achievementmotivation, Journal of Personality and Social Psychology, 72(1), 218–232.

Elliot, A. J. & Harackiewicz, J. M. (1996) Approach and avoidance achievement goals andintrinsic motivation: a mediational analysis, Journal of Personality and Social Psychology,70(3), 461–475.

Elliot, A. J. & McGregor, H. (2001) A 2 × 2 achievement goal framework, Journal of Personalityand Social Psychology, 76(4), 628–644.

Finch, J. (1987) The vignette technique in survey research, Sociology, 21(1), 105–114.Gould, D. (1996) Using vignettes to collect data for nursing research studies: how valid are the

findings? Journal of Clinical Nursing, 5(4), 207–212.Grant, H. & Dweck, C. S. (2003) Clarifying achievement goals and their impact, Journal of Person-

ality and Social Psychology, 85(3), 541–553.Harackiewicz, J. M., Barron, K. E., Pintrich, P. R., Elliot, A. J. & Thrash, T. M. (2002a) Revision

of achievement goal theory: necessary and illuminating, Journal of Educational Psychology,94(3), 638–645.

Harackiewicz, J. M., Barron, K. E., Tauer, J. M., Carter, S. M. & Elliot, A. J. (2000) Short-termand long-term consequences of achievement goals: predicting interest and performance overtime, Journal of Educational Psychology, 92(2), 316–330.

Harackiewicz, J. M., Barron, K. E., Tauer, J. M. & Elliot, A. J. (2002b) Predicting success incollege: a longitudinal study of achievement goals and ability measures as predictors of

248 J. W. Young

interest and performance from freshman year through graduation, Journal of EducationalPsychology, 94(3), 562–575.

Heggestad, E. D., Morrison, M., Reeve, C. L. & McCloy, R. A. (2006) Forced-choice assessmentsof personality for selection: evaluating issues of normative assessment and faking resistance,Journal of Applied Psychology, 91(1), 9–24.

Hughes, R. (1998) Considering the vignette technique and its application to a study of druginjecting and HIV risk and safer behaviour, Sociology of Health and Illness, 20(3), 381–400.

Hughes, R. & Huby, M. (2002) The application of vignettes in social and nursing research, Journalof Advanced Nursing, 37(4), 382–386.

Kyllonen, P.C., Walters, A. M. & Kaufman, J. C. (2005) Noncognitive constructs and their assess-ment in graduate education: a review, Educational Assessment, 10(3), 153–183.

Linnenbrink, E. A. & Pintrich, P. R. (2000) Multiple pathways to learning and achievement: therole of goal orientation in fostering adaptive motivation, affect, and cognition, in: C. Sansone& J. M. Harackiewicz, (Eds) Intrinsic and extrinsic motivation: the search for optimal motivationand performance (New York, Academic Press), 195–227.

Midgley, C., Kaplan, A., Middleton, M., Maehr, M. L., Urdan, T., Anderman, L., Anderman, E.& Roeser, R. (1998) The development and validation of scales assessing students’ achieve-ment goal orientations, Contemporary Educational Psychology, 23(2), 113–131.

Nichols, J. G. (1984) Achievement motivation: conceptions of ability, subjective experience, taskchoice, and performance, Psychological Review, 91(3), 328–346.

Pintrich, P. R. (2000a) An achievement goal theory perspective on issues in motivation terminol-ogy, theory, and research, Contemporary Educational Psychology, 25(1), 92–104.

Pintrich, P. R. (2000b) Multiple goals, multiple pathways: the role of goal orientation in learningand achievement, Journal of Educational Psychology, 92(3), 544–555.

Pintrich, P. R. (2000c) The role of goal orientation in self-regulated learning, in: M. Boekaerts, P.R. Pintrich, & M. Zeidner (Eds) Handbook of self-regulation: theory, research, and applications(San Diego, CA, Academic Press), 451–502.

Rahman, N. (1996) Caregivers’ sensitivity to conflict: the use of vignette methodology, Journal ofElder Abuse and Neglect, 8(1), 35–47.

Urdan, T. C. (1997) Achievement goal theory: past results, future directions, in: M. L. Maehr &P. R. Pintrich (Eds) Advances in motivation and achievement (Greenwich, CT, JAI Press),99–141.

Wagner, R. K. & Sternberg, R. J. (1986) Tacit knowledge and intelligence in the everyday world,in: R. J. Sternberg & R. K. Wagner (Eds) Practical intelligence: nature and origins of competencein the everyday world (Cambridge, Cambridge University Press), 51–83.

Wolters, C. A. (2004) Advancing achievement goal theory: using goal structures and goal orienta-tions to predict students’ motivation, cognition, and achievement, Journal of EducationalPsychology, 96(2), 236–250.

Young, J. W. (2001) Differential validity, differential prediction, and college admission testing: acomprehensive review and analysis (College Board Research Report No. 2001–6) (New York,College Entrance Examination Board).

Young, J. W. & Robustelli, S. L. (2004, April) Development and validation of the Multiple GoalsTheory Measure of academic motivation, in: the symposium, ‘Noncognitive measures incollege admissions’. Symposium presented at the annual meeting of the American EducationalResearch Association, San Diego, CA.


APPENDIX A Instructions for the Multiple Goals Theory Measure and Sample Vignette

Instructions: The following survey includes stories pertaining to high schoolacademics and settings. Please read each story and rate how likely or unlikely youare to endorse that option. Please circle the number that most accurately reflects howlikely your reaction would be for each situation. Be sure to circle one response foreach item. Remember, there are no right or wrong answers, so please answer ashonestly and accurately as possible.

Sample Question:

Today you have a history exam. You have spent a lot of time studying for it. You havereviewed your notes, read the material, and even participated in group study sessions tobetter prepare you for taking it. What perspective would you enter the exam with?

I’ll do the best I can. The purpose of an exam is to learn from it.

I am sure I will get one of the best grades in the class.

I hope to perform up to my standards.

I hope I don’t get the lowest grade in the class because others will think I am notsmart.

1 2 3 4 5

Not likely A bit likely Somewhat likely Likely Very likely

1 2 3 4 5


1 2 3 4 5


1 2 3 4 5


predicting college grades: the value of achievement goals in supplementing ability measures

Documents