phd experience and subsequent outcomes: a look at self-perceptions of acquired graduate attributes...
TRANSCRIPT
This article was downloaded by: [Northeastern University]On: 17 November 2014, At: 20:14Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954 Registeredoffice: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK
Studies in Higher EducationPublication details, including instructions for authors andsubscription information:http://www.tandfonline.com/loi/cshe20
PhD experience and subsequentoutcomes: a look at self-perceptionsof acquired graduate attributes andsupervisor supportMichael J. Platow aa Department of Psychology , The Australian National University ,Canberra , ACT , 0200 , AustraliaPublished online: 26 Apr 2011.
To cite this article: Michael J. Platow (2012) PhD experience and subsequent outcomes: a lookat self-perceptions of acquired graduate attributes and supervisor support, Studies in HigherEducation, 37:1, 103-118, DOI: 10.1080/03075079.2010.501104
To link to this article: http://dx.doi.org/10.1080/03075079.2010.501104
PLEASE SCROLL DOWN FOR ARTICLE
Taylor & Francis makes every effort to ensure the accuracy of all the information (the“Content”) contained in the publications on our platform. However, Taylor & Francis,our agents, and our licensors make no representations or warranties whatsoever as tothe accuracy, completeness, or suitability for any purpose of the Content. Any opinionsand views expressed in this publication are the opinions and views of the authors,and are not the views of or endorsed by Taylor & Francis. The accuracy of the Contentshould not be relied upon and should be independently verified with primary sourcesof information. Taylor and Francis shall not be liable for any losses, actions, claims,proceedings, demands, costs, expenses, damages, and other liabilities whatsoeveror howsoever caused arising directly or indirectly in connection with, in relation to orarising out of the use of the Content.
This article may be used for research, teaching, and private study purposes. Anysubstantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,systematic supply, or distribution in any form to anyone is expressly forbidden. Terms &Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions
Studies in Higher Education
ISSN 0307-5079 print/ISSN 1470-174X online© 2012 Society for Research into Higher Education
http://www.tandfonline.com
PhD experience and subsequent outcomes: a look at self-perceptions of acquired graduate attributes and supervisor support
Michael J. Platow*
Department of Psychology, The Australian National University, Canberra, ACT 0200, AustraliaTaylor and FrancisCSHE_A_501104.sgm10.1080/03075079.2010.501104Studies in Higher Education0307-5079 (print)/1470-174X (online)Original Article2011Society for Research into Higher Education0000000002011Dr. [email protected]
The literature examining graduate attributes remains unclear regarding their valueto work within and outside of the university context. The current article soughtclarity through a quantitative analysis of the relationship between self-perceptionsof PhD-related graduate attribute acquisition and both objective (e.g. productivity)and subjective outcomes (i.e. evaluations of PhD experience). The perceivedacquisition of graduate attributes was related to post-PhD productivity andsubjective evaluations, but not duration of PhD study, productivity during PhDstudy, time seeking employment post-PhD or current gross salary. Perceptions ofsupervisor support, as well as a variety of demographic variables, were also relatedto several of the current outcome measures. Overall, the results speak to the rolethat university graduate education has in providing for the development ofattributes that can be successfully brought to fruition in post-graduationemployment through enhanced productivity.
Keywords: graduate skills; generic skills; doctoral education; doctoral supervision;graduate education
Introduction
A small but persistent debate is waging over the meaning and validity of generic qual-ities and skills, that supposedly develop alongside (if not interdependently with) thedisciplinary content of higher education, and yet are not isomorphic with that content(e.g. Barnett 2004; Barrie 2006; Craswell 2007). These qualities and skills include,among other things, the ability to think critically, to contribute to scholarship, to takeindividual initiative and to work as a member of a team (e.g. Barrie 2004; Gilbert et al.2004; Western et al. 2007a). The development of these qualities and skills, collec-tively referred to as ‘graduate attributes’, is increasingly becoming the raison d’êtrefor continued university education in its most common form. Indeed, if universities donot promote the development of these attributes, they need only to (indeed, ought onlyto) provide vocational education rather than traditional liberal arts or theory-baseddegrees (Barrie 2004; Gilbert et al. 2004). Universities are, thus, finding it increas-ingly necessary both to articulate these attributes and to demonstrate their presenceand usefulness within their graduates.
This article contributes to the research on graduate attributes by providing a quan-titative assessment of the relationship between graduates’ self-perceptions of theacquisition of these attributes during their PhD studies and a variety of outcomes, both
*Email: [email protected]
Vol. 37, No. 1, February 2012, 103–118
http://dx.doi.org/10.1080/03075079.2010.501104
Dow
nloa
ded
by [
Nor
thea
ster
n U
nive
rsity
] at
20:
14 1
7 N
ovem
ber
2014
during and after the award of their degree. In doing so, the potential moderation ofthese relationships (if any exist) by perceptions of PhD-supervisor support is alsoexamined (cf. Sinclair 2004); any positive consequences derived from the develop-ment of graduate attributes may well be observed only under conditions of relativelyhigh supervisor support.
Graduate attributes: defining, teaching, assessing
A variety of models of graduate attributes are presented in the higher education liter-ature (e.g. Barrie 2004; Gilbert et al. 2004), in government statements (e.g. AustralianGovernment, n.d.), and on university web pages (e.g. University of Cambridge 2009).These models tend to vary as a function of the manner in which the attributes weredetermined, either being inferred through empirical investigation or derived from(often unstated) personal, social and political goals and values. Gilbert et al. (2004)outline, for example, lists of attributes provided by several educational bodies in theUnited States, the UK and Australia. Included on these lists are attributes such as‘demonstrate research skills and techniques’ and ‘ability to see oneself as a scholar-citizen’ (377). Although potentially useful for the practitioner in their concrete, dot-pointed nature, such lists often fail to be accompanied by an overarching theme, or setof dimensions or principles. In short, there is often no formal theoretical frameworkfrom which these attributes are derived (Manathunga and Lant 2006). The absence ofsuch a framework allows the field to be complicated by ever increasing lists, oftenderived in ad hoc manners to satisfy, inter alia, specific political and economicagendas. Recently, however, Cumming et al. (2009) reviewed a variety of such lists,and provided some order by dichotomizing them into the two broad categories of‘academic skills’ and ‘employability skills’ (7). According to their analysis, theformer includes attributes associated with specific disciplinary knowledge, whereasthe latter includes attributes associated with teamwork, effective communication andtime management.
In contrast to mere listing, Barrie (2004, 2006) derived a four-level, hierarchicalmodel of graduate attributes from in-depth analyses of semi-structured interviewswith academics across a variety of disciplines. At the base level of this model areprerequisite skills that are assumed to be developed prior to university entrance(e.g. language proficiency, basic quantitative skills). Although these can be under-stood more as attributes of primary and secondary school, they remain essential forsuccessful tertiary education (see Barrie [2006, 225–26] for a further discussion ofthis aspect of his model and, in particular, the position and role of these prerequi-site skills in his graduate-attribute model). At the second level are skills andattributes that complement, but do not (necessarily) derive directly from, disciplin-ary content. These include skills such as time management and basic composition.Students can gain these second-level skills through independent courses and work-shops that may sit beside standard disciplinary education, or from activities outsideformal education.
Acquisition of the third-level and fourth-level skills and attributes represents aqualitative leap from the discrete, additive nature of the first two levels. Barrie (2006,224) refers to this leap as ‘transformative’. He sees transformative skills and attributesas interacting with disciplinary knowledge rather than simply adding to it. In thismanner, acquiring third-level skills and attributes means that students can use andapply disciplinary knowledge within the discipline itself. And at the fourth-level of
M.J. Platow104
Dow
nloa
ded
by [
Nor
thea
ster
n U
nive
rsity
] at
20:
14 1
7 N
ovem
ber
2014
Barrie’s hierarchy lay complex, ‘enabling’ skills that ‘provide a reusable frameworkthat enables students/graduates to acquire and shape new knowledge as required –even in the context of other disciplines’ (230). The value of this type of model is thatit provides a meaningful framework within which students, educators and employerscan understand and strategically utilize otherwise unstructured and increasinglycomplex lists.
Adding further to the graduate attributes debate are the arguments of authors whocall for the addition of new attributes (e.g. Bridgstock 2009), clarification of currentmodels (Green, Hammer, and Star 2009) or complete reformulations of currentmodels (e.g. Barnett 2004; Craswell 2007). Despite their differences, however, allauthors seem to agree that some broad skills, knowledge, abilities or attributes that areindependent of disciplinary content (although that may develop through discipline-specific activities; Green, Hammer, and Star 2009; Manathunga and Lant 2006)remain valuable outcomes of tertiary education. This basic assumption has led othereducators to construct programs intended to develop these attributes. The constructionof these programs often begins with the goal of developing specific attributes, such asproblem solving (Geenan, Humphreys, and McIlveen 1997; Manathunga 2004;O’Brien and Hart 1999; Trevleaven and Voola 2008) or publishing skills (e.g. Cargill2004); some programs even involve supplementing university education with on-the-job experience (e.g. Crebert et al. 2004; Muldoon 2009). In most cases, authors haveendeavoured to measure the relative success of their programs, reporting at leastmodest achievement of their goals. However, to date, the measures tend to focus onstudents’ subjective evaluations, such as satisfaction ratings or perceived value of thelearning content.
At least two studies have tried to evaluate the relationship between the acquisi-tion of graduate attributes and actual behaviour in the workplace. In one, Bermanand Pitman (2010) surveyed university staff members employed in general (i.e. notacademic) positions. Critical to their research was that these employees all hadearned PhDs, so the supposed development of graduate attributes was high.Through their qualitative analyses, Berman and Pitman concluded that there was,indeed, a high level of successful use of generic skills being employed in the work-place that had developed during undergraduate and postgraduate education. Thesuccessful use of these skills was so high that the authors pondered why universities‘do not actively target the products of their own system for professional roles’ (10).The second study, however, presents a somewhat less favourable impression. In thisstudy, Manathunga, Pitt, and Critchley (2009) surveyed people who had completedtheir PhDs in science. Respondents were asked to indicate the degree to which skillsdeveloped during their PhD studies ‘prepared’ (98) them for their current employ-ment. Although the majority thought that they did have sufficient skills, the actualpercentage was as low as 58% (in one sample). Moreover, many respondents indi-cated needing further development of complementary skills (e.g. time management,goal setting).
The state of affairs in the graduate attributes literature is, thus, less than clear interms of identifying the actual attributes and the relative value of these attributes towork, both within and outside of the university context. Indeed, there is even disagree-ment over whether these ought to be referred to as, for example, graduate attributes(as I have chosen to call them; e.g. Manathunga 2004), generic skills (e.g. Gilbertet al. 2004) or transferable skills (e.g. Cryer 1998). In an attempt to seek some clarity,the primary goal in the current article, as noted above, was to evaluate quantitatively
Studies in Higher Education 105
Dow
nloa
ded
by [
Nor
thea
ster
n U
nive
rsity
] at
20:
14 1
7 N
ovem
ber
2014
the relationship between a set of graduate attributes and both objective and subjectiveoutcome measures.
On supervisor support
Having recognized the potential importance of graduate attributes, however, it is clearfrom the extant literature that assessing their role alone is insufficient. There is clearevidence now that, when starting PhD studies, students are often unclear about whatthey are supposed to do (Pole et al. 1997), and a disparity exists between students’ andsupervisors’ conceptions of research (Bills 2004; Kiley and Mullins 2005; Meyer,Shanahan, and Laugksch 2005). There is, thus, now a broad consensus in the literaturethat appropriate supervisor support throughout PhD studies is essential (e.g. Frischerand Larsson 2000; Gurr 2001; Pearson and Brew 2002), with empirical evidencedemonstrating the value of such support to PhD success (Sinclair 2004).
This empirical evidence also suggests that, although not necessarily formally taught(e.g. in a classroom setting), the development of graduate attributes is likely to beenhanced through higher levels of supervisor support (such as helping students developrelevant skills and knowledge; e.g. Sinclair 2004). It is in this way that the positiveoutcomes derived from the development of graduate attributes may be observed onlyunder conditions of relatively high supervisor support. A secondary goal in the currentarticle was, thus, to examine the independent and moderating role of perceived levelsof supervisor support on the various outcome measures currently employed.
Method
Respondents
The original sample contained 2020 respondents who had completed a questionnairedesigned and administered by Western et al. (2007a, b; 2010). The response rate forthe original sample was 38%, with the sample being largely representative of thepopulation (see Western et al. 2007a). For the current analyses, however, only respon-dents who had completed a PhD and who provided complete data sets for the currentvariables of interest were included. These criteria reduced the usable sample to 1258respondents (576 women), with a median age of 38 years. A total of 202 respondentsindicated that they were not Australian citizens, and seven respondents self-identifiedas Indigenous Australians. There were 888 respondents (71%) who reported being‘wholly or mainly full-time’ students during their PhD studies; 1008 (80%) respon-dents reported being employed during their PhD studies, while 1086 (86%) reportedhaving some form of scholarship. As regards number of supervisors, 523 respondentsreported having only one supervisor, 562 reported having two supervisors, 173reported having ‘three or more’ supervisors, and 657 respondents reported being partof a research group during their PhD studies. In terms of broad disciplinary categories,524 respondents were in the natural or physical sciences, 158 were in engineering orinformation technology, 48 were in other hard applied sciences (e.g. architecture), 140were in health and 388 were in the social sciences, humanities and arts.
Materials
A subset of items obtained from Western et al.’s (2007b) larger questionnaire wasused in the current study; these items represented three broad categories.
M.J. Platow106
Dow
nloa
ded
by [
Nor
thea
ster
n U
nive
rsity
] at
20:
14 1
7 N
ovem
ber
2014
Measures of objective and subjective outcomes
Five measures were sampled as objective reflections of PhD success: (1) actual duration(in months) of PhD study; (2) number of months out of paid employment (but looking foremployment) from PhD submission; (3) current gross income, measured on a 10-pointscale (1 = ‘1–$9,999’, 10 = ‘$150,000 or more’); (4) total output produced during PhDstudies; and (5) total output since PhD completion. For the final two measures, respon-dents indicated the total number of each of the following forms of output that they, atminimum, shared responsibility for: (a) authored books; (b) edited books; (c) chaptersin books; (d) refereed articles; (e) research/technical reports; (f) conference papers; (g)patents; (h) software products; (i) compositions; (j) exhibitions; and (k) performances.Because of the diversity of disciplines currently sampled and the diversity of outputs,a simple sum was calculated across all output forms for each respondent for the currentdependent variables; other summary statistics placing differential weights on outcomeswere currently infeasible given the absence of, for example, importance ratings of thevarious outcomes from the research respondents (i.e. a specific basis for differentialweighting). The term ‘objective’ is used in describing these outcomes primarily to differ-entiate them from attitudinal outcomes (described below); I am mindful, of course, thatthese were all measured via self-report instead of, say, a third-party (or archival) obser-vation, and this must be borne in mind in any inferences made.
A single subjective outcome measure was employed based upon two itemsmeasuring perceived usefulness of the PhD for respondents’ current work activities:(1) ‘Overall, how useful has your PhD training been to your current/most recent workactivities?’ and (2) ‘Overall, how useful has your PhD training been to your careerafter your PhD?’ Responses were coded on a scale from 1 (‘not useful at all’) to 5(‘very useful’). These two items were highly and significantly correlated (r = .81,p < .001), so the mean of them formed the final dependent variable.
Measures of PhD experiences
Two sets of items were employed as measures of respondents’ PhD experiences. Thefirst set of items (see Table 1) formed a single scale representing a global measure ofself-perceptions of graduate attributes acquired during respondents’ PhD studies.Although varied, these items together had a high level of internal reliability (α = .90);because of this strong empirical relationship, attempts to differentiate between theitems either along Barrie’s (2004) or Cumming et al.’s (2009) models were eschewed.
A mean of all the items was, thus, calculated for the analyses as a measure ofperceptions of the acquisition of global graduate attributes (see, however, Westernet al. 2010). Respondents were specifically asked, ‘During your PhD, to what extentdid you acquire the following characteristics?’ (emphasis in original).
The second set of items (see Table 2) also formed a reliable scale (alpha = .84),and represented participants’ perceptions of their supervisors’ support. All items wereprefaced with the statement, ‘To what extent did your principal supervisor’. Again, amean of these items was calculated for the analyses. Responses to both sets of itemswere measured on a scale from 1 (coded ‘not at all’) to 5 (‘to a great extent’).
Measures of demographic variables
Finally, a series of demographic variables likely to be related to the outcome variables(Sinclair 2004; Wright and Cochrane 2000) was employed as a control in all analyses.
Studies in Higher Education 107
Dow
nloa
ded
by [
Nor
thea
ster
n U
nive
rsity
] at
20:
14 1
7 N
ovem
ber
2014
These variables were: (1) respondents’ sex (0 = female, 1 = male); (2) respondents’age when starting their PhD; (3) whether respondents held a scholarship during theirPhD studies (0 = no, 1 = yes); (4) whether respondents were employed duringtheir PhD studies (0 = no, 1 = yes); (5) respondents’ enrolment status (0 = part-time,1 = full-time); (6) whether respondents were Indigenous Australians (0 = Indigenous,1 = non-Indigenous); (7) whether participants completed their PhD as part of aresearch group (0 = no, 1 = yes); (8) the number of supervisors respondents had (1, 2,or 3, with 3 representing ‘three or more’); and (9) respondents’ citizenship status (0 =resident, 1 = citizen). Each of the broad disciplinary categories was also dummy-coded, using ‘other hard applied sciences’ as the reference group.
Procedure
All students who had earned degrees from the top eight research-intensive Australianuniversities between 1999 and 2001 were eligible to participate in this study; self-reports of PhD thesis submission dates, however, ranged from 1989 to 2004 (thevalues pre-dating 1999 may have been response errors; however, I use the providedresponses in the analysis below on post-PhD productivity). Data were collected in2006 via a structured survey administered over the Internet or distributed by post.
Table 1. Items measuring self-perceptions of the acquisition of graduate attributes duringPhD studies.
1. A capacity for creativity and innovation2. A capacity to contribute to the development of professional practice3. An ability to contribute to scholarship in your discipline or field4. An ability to solve problems successfully5. An ability to take individual initiative6. An ability to think critically7. An ability to work as a member of a team8. An ability to work in an interdisciplinary context9. An understanding of ethical values in research10. Assertiveness11. Data analysis skills/ability to make sense of evidence12. Financial management skills13. Independent research skills14. Leadership skills15. Skill in grant writing16. Skill in oral communication17. Skill in project management18. Skill in writing reports/articles/books19. Teaching skills20. Up-to-date knowledge and skills about methodological issues21. Up-to-date knowledge and skills in your substantive field
Table 2. Items measuring perceived PhD supervisor support.
1. Help you develop professional relationships with others in the field2. Encourage you to publish your work3. Encourage you to give conference papers4. Assist you in preparing proposals for funding5. Assist you in gaining employment6. Help you develop relevant skills and knowledge
M.J. Platow108
Dow
nloa
ded
by [
Nor
thea
ster
n U
nive
rsity
] at
20:
14 1
7 N
ovem
ber
2014
Results
The very nature of the data set currently employed allows for only correlational infer-ences to be made. Although particular predictor and outcome variables are employedin the analyses below, I recognize, of course, that the causal direction for the key vari-ables of interest (graduate attributes and supervisor support) could actually work in theopposite direction; for example, respondents who have high post-PhD productivitymay infer a relatively high level of acquisition of graduate attributes. Despite thisnecessary limitation of using cross-sectional data, the analyses remain informative,and bear upon the key educational processes currently of interest.
For each of the outcome variables, either a separate hierarchical linear regressionor a binomial logistic regression (depending on the nature of the outcome variable)was conducted. For the first step in each regression, all of the demographic variablesdescribed above were entered. For the second step, the two variables measuringperceptions of the PhD experiences were entered. Finally, to identify whether theimpact on the outcome variables of the acquisition of global graduate attributes wasmoderated by the level of perceived supervisor support, the scales measuring thesetwo constructs were centred (Aiken and West 1991), and an interaction term wascalculated; this interaction term was entered into the third step of the analyses.
Although some of the zero-order correlations between the predictor variables weresignificant, none was above .56 (which was between two of the dummy-codeddisciplinary categories). Of note was the negative relationship between being in aresearch group and studying in the social sciences, humanities and arts (r = −.52); andthe positive relationship between enrolment status and being awarded a scholarship,with full-time students more likely to have a scholarship (r = .44). None of the corre-lations were sufficiently large to raise concerns about multicollinearity (Tabachnickand Fidell 2007). Table 3 presents the means and standard deviations for each ofthe outcome variables. Table 4 presents a summary of the regression model tests foreach analysis described below, and Table 5 summarizes the statistically significantpredictors in each analysis.
Duration of PhD study
In analysing the duration of PhD study, both the first and second step of the analysiswere significant; there was a significant R2-change from Step 1 to Step 2, but not fromStep 2 to Step 3, yielding a final two-step model only. Duration of PhD study waslonger when respondents were younger when they started, when they were IndigenousAustralians, and when they were employed during their PhD study. Part-time status
Table 3. Means and standard deviations for each of the outcome variables.
Outcome variable Mean Standard deviation
Duration of PhD study (in months) 57.93 20.64Number of months out of paid employment 1.86 5.58Current gross salary* 6.11 1.67Total amount of output produced during PhD studies 12.23 17.47Total amount of output produced after PhD studies 22.46 30.64Subjective evaluations of PhD experience 4.24 0.94
*On the response scale, a value of 6 = $60,000–$79,999.
Studies in Higher Education 109
Dow
nloa
ded
by [
Nor
thea
ster
n U
nive
rsity
] at
20:
14 1
7 N
ovem
ber
2014
Tabl
e 4.
Sum
mar
y of
reg
ress
ion
mod
el t
ests
for
eac
h ex
amin
ed o
utco
me
vari
able
.
Dur
atio
n of
P
hD s
tudy
Une
mpl
oyed
upo
n P
hD s
ubm
issi
onN
umbe
r of m
onth
s (>
0)
wit
hout
pai
d em
ploy
men
tC
urre
nt g
ross
sa
lary
Tot
al o
utpu
t du
ring
PhD
Tot
al o
utpu
t po
st-P
hD
Eva
luat
ions
of
PhD
ex
peri
ence
Ste
p/B
lock
1F
(13,
1244
) =
29
.68*
**χ2 (1
3, n
= 1
,258
) =
30
.88*
*F
(13,
291)
=
2.63
**F
(13,
1244
) =
9.
91**
*F
(13,
1244
) =
7.
93**
*F
(15,
1242
) =
10
.79*
**F
(13,
1244
) =
1.
98*
Ste
p/B
lock
2F
(15,
1242
) =
27
.10*
**χ2 (1
5, n
= 1
,258
) =
62
.43*
**—
—F
(15,
1244
) =
7.
67**
*F
(17,
1240
) =
10
.62*
**F
(15,
1242
) =
24
.03*
**S
tep/
Blo
ck 3
——
——
——
F(1
6,12
41)
=
23.8
0***
R2 /f
it-c
hang
e:
step
/blo
ck 1
to
2F
(2,1
242)
=
8.12
***
χ2 (2, n
= 1
,258
) =
31
.54*
**F
(2,2
89)
=
0.74
, ns
F(2
,124
2) =
0.
83, n
sF
(2,1
242)
=
5.60
**F
(2,1
240)
=
8.41
***
F(2
,124
2) =
16
3.97
***
R2 /f
it-c
hang
e:
step
/blo
ck 2
to
3F
(1,1
241)
=
0.58
, ns
χ2 (1, n
= 1
,258
) =
0.
06, n
s—
—F
(1,1
241)
=
2.18
, ns
F(1
,123
9) =
.5
4, n
sF
(1,1
241)
=
15.9
9***
ˆR2 f
or f
inal
mod
el.2
4.0
7.0
7.0
8.0
7.1
2.2
3
Not
e: P
rese
nted
dat
a ar
e fo
r fin
al m
odel
s on
ly.
ˆFor
lin
ear
regr
essi
ons,
thi
s is
the
adj
uste
d R
2 ; fo
r th
e lo
gist
ic r
egre
ssio
n, t
his
is N
agel
kerk
e R
2 . *p
< .0
5; *
*p <
.01;
***
p <
.001
.
M.J. Platow110
Dow
nloa
ded
by [
Nor
thea
ster
n U
nive
rsity
] at
20:
14 1
7 N
ovem
ber
2014
Tabl
e 5.
Sum
mar
y of
sta
tist
ical
ly s
igni
fica
nt (
and
mar
gina
lly s
igni
fica
nt)
pred
icto
rs o
f ea
ch e
xam
ined
out
com
e va
riab
le.
Dur
atio
n of
P
hD s
tudy
Une
mpl
oyed
up
on P
hD
subm
issi
on
Num
ber
of m
onth
s (>
0)
wit
hout
pai
d em
ploy
men
tC
urre
nt
gros
s sa
lary
Tot
al o
utpu
t du
ring
PhD
Tot
al o
utpu
t po
st-P
hD
Eva
luat
ions
of
PhD
ex
peri
ence
ˆSex
(1
= m
ale)
β =
.20*
**β
= .0
8**
β =
.07*
*ˆA
ge a
t st
art
of P
hDβ
= −
.07*
β =
.19*
*β
= .1
4***
Aus
tral
ian
citi
zens
hip
β =
.05ˆ
ˆβ
= .0
8**
Non
-Ind
igen
ous
Aus
tral
ian
β =
−.0
7*E
mpl
oyed
dur
ing
PhD
stu
dies
β =
.08*
*β
= .0
8**
Sch
olar
ship
β =
−.0
9**
β =
−.1
0**
ˆEnr
olm
ent
stat
us (
1 =
ful
l ti
me)
β =
−.4
4***
β =
−.1
3*β
= −
.10*
*N
atur
al a
nd p
hysi
cal
scie
nces
B =
.82*
Wor
ked
in r
esea
rch
grou
p (1
= y
es)
β =
.12*
**N
umbe
r of
sup
ervi
sors
β =
.10*
**G
radu
ate
attr
ibut
esβ
= .1
0**
β =
.37*
**ˆS
uper
viso
r su
ppor
t β
= −
.11*
**B
= −
.35*
**β
= .1
7***
Gra
duat
e at
trib
utes
x s
uppo
rtβ
= −
.10*
**
ˆThe
se p
redi
ctor
s yi
elde
d si
gnifi
cant
rel
atio
nshi
ps w
ith
at l
east
thr
ee o
f th
e se
ven
outc
ome
vari
able
s. ˆ
ˆp =
.05;
*p
< .0
5; *
*p <
.01;
***
p <
.001
.
Studies in Higher Education 111
Dow
nloa
ded
by [
Nor
thea
ster
n U
nive
rsity
] at
20:
14 1
7 N
ovem
ber
2014
also increased the total amount of months to complete the PhD, but the currentoutcome variable is coded in calendar months, so this effect simply reflects respon-dents’ enrolment status. There was also a marginally significant effect for citizenshipstatus, with Australian citizens taking longer to complete their PhD than non-citizens.The effect of graduate attributes was non-significant (β = .02, p = .52). However, therewas a significant negative relationship between perceived supervisor support andduration of study; the more perceived support respondents reported receiving fromtheir supervisors, the fewer months they took to complete their PhD study.
Number of months out of paid employment
An examination of the distribution of the number of months out of paid employmentfollowing PhD thesis submission revealed that 75.76% of all respondents had jobsimmediately following their thesis submission. Clearly, the skewed nature of thedistribution prohibited use of a standard linear regression for the analysis. The datawere, thus, recoded to indicate either no time out of employment (value of 0) or sometime out of employment (value of 1). These data were analysed using a hierarchicalbinomial logistic regression. Results indicated that the models at block 1 and block 2were significant, with block 2 improving the fit over block 1, but with block 3 offeringno substantial improvement, yielding a final two-block model. Two statisticallysignificant effects emerged from this analysis: (1) respondents studying in the naturalor physical sciences were less likely to have a job immediately upon submission oftheir PhD thesis than respondents in other disciplines, and (2) respondents who had nodelay in obtaining post-submission employment reported greater supervisor support.
By placing into a single category those respondents (n = 305) who did not haveimmediate employment post-PhD submission, the above analysis ignored a range ofactual variability in their duration of employment seeking; this reported range wasbetween 1 and 89 months. To identify variables related to this variability, a standardhierarchical linear regression among this subset was conducted. Although the first stepin this analysis was significant, there was no significant R2-change to the second step,yielding only a single-step model. Respondents who were older at the start of theirPhD studies and respondents who studied part-time had greater delays in obtainingemployment post-submission.
Current gross salary
The first step in the analysis of current gross salary was significant, and there was nosignificant R2-change to the second step, yielding only a single-step model. Highersalaries were earned by men than women, Australian citizens than non-citizens,students who were employed during their PhD studies, and people who did not havea scholarship.
Total amount of output produced during PhD studies
In the analysis of total amount of output produced during PhD studies, both the firstand second steps were significant, with a significant increase in R2 from the first to thesecond step, but not from the second to the third, yielding a final two-step model.Respondents working in research groups during their PhD studies were more produc-tive than those not working in research groups. Surprisingly, however, respondents who
M.J. Platow112
Dow
nloa
ded
by [
Nor
thea
ster
n U
nive
rsity
] at
20:
14 1
7 N
ovem
ber
2014
did not have scholarships were more productive than those who did, and part-timestudents were more productive than full-time students. Finally, respondents who startedtheir PhD studies at an older age were more productive than those who started at ayounger age. Despite the significant improvement in R2 with the addition of perceivedacquisition of general graduate attributes and perceived supervisor support, each ofthese predictors only approached traditional levels of significance (β = .06, p = .07; andβ = .06, p = .09, respectively).
Total amount of output produced after PhD studies
In analysing the total amount of output produced after PhD studies, I was mindful ofthe significant correlation between this variable and the total amount of outputproduced during PhD studies (r = .27, p < .001). Although this correlation is small, Iwanted to be confident that any conclusions made regarding post-PhD productivitywere not simply based on productivity during PhD studies. Therefore, in the currentanalysis, productivity during PhD studies was added as an additional predictor in step1 of the analysis. In addition, respondents who had submitted their PhD theses morerecently were likely to have produced less output between submission and completionof the current questionnaire than respondents who had submitted their theses earlier.Therefore, I added year and month (as a fraction) of thesis submission as yet anotherpredictor in step 1 of the current analysis to control for this effect.
Both the first and second steps in the analyses were significant, with a significantincrease in R2 from the first to the second step, but not from the second to the third,yielding a final two-step model. As anticipated, the greater the respondents’ produc-tivity as PhD students, the greater was their post-PhD productivity (β = .26, p < .001).Also as anticipated, the more recently respondents had submitted their theses, thelower was their post-PhD productivity (β = −.16, p < .001). Of greater interest, post-PhD productivity was higher for men than women, and for respondents who had more,rather than fewer, PhD supervisors. Finally, and of greatest interest to the currentresearch, greater levels of perceived acquisition of general graduate attributes wereassociated with higher levels of post-PhD productivity.
Subjective evaluations of PhD experience
In the analysis of respondents’ subjective evaluations of the usefulness of their PhDexperiences, all three steps of the regression were significant, as was the change fromthe first step to the second, and the second step to the third, yielding a final three-stepmodel. Men reported greater perceived usefulness of their PhD experience than didwomen. Importantly, perceptions of supervisor behaviour and the acquisition ofglobal graduate attributes were significantly and positively related to overallperceived usefulness of the PhD experience. Finally, the interaction between supervi-sor behaviour and graduate attributes was significant. This interaction is displayed inFigure 1; plotted values are one standard deviation above and below the mean for eachof the two predictors (Aiken and West 1991), and are centred at mean values of theother predictors in the regression. As can be seen, and counter to expectations, theacquisition of higher levels of graduate attributes seems to be a means of overcomingthe negative impact on perceptions of PhD usefulness of lower levels of supervisorsupport.
Studies in Higher Education 113
Dow
nloa
ded
by [
Nor
thea
ster
n U
nive
rsity
] at
20:
14 1
7 N
ovem
ber
2014
Figure 1. Moderating effects of supervisor behaviour on the relationship between perceptions of the acquisition of global graduate attributes and subjective evaluations of the usefulness of the PhD experience for current employment.Discussion
The goal of the present article was to observe and measure the (potential) relationshipbetween self-perceptions of the acquisition of graduate attributes and a variety ofobjective and subjective outcomes commonly employed as indices of PhD success. Ascurrently measured, a positive role of graduate attributes was certainly present,although definitely limited. The perceived acquisition of these attributes was relatedonly to post-PhD productivity and subjective evaluations of the value of the PhDexperience; no relationship was observed with completion times, PhD productivity,job acquisition or current salary. In the light of this limited role, one might reasonablyseek to question the claims made by universities regarding the value of the educationthey provide. The data, however, remain unproblematic for these claims: althoughthese attributes, as currently observed, were unrelated to behaviour as a student (e.g.PhD productivity), the student experience provided for their development, so that itwas in the post-PhD environment in which the consequences of these attributes cameto fruition. In this manner, the graduate attributes begin to take the form of the qualityof ‘graduateness’ (e.g. Holmes and Miller 2000; Glover, Law, and Youngman 2002),in which higher education provides the appropriate developmental environment for,inter alia, attributes valuable in the workplace (Griesel 2003).
Also of interest in the current findings is that supervisor support moderated therole of graduate attributes only on subjective evaluations, and in a way counter to
Figure 1. Moderating effects of supervisor behaviour on the relationship between perceptionsof the acquisition of global graduate attributes and subjective evaluations of the usefulness ofthe PhD experience for current employment.
M.J. Platow114
Dow
nloa
ded
by [
Nor
thea
ster
n U
nive
rsity
] at
20:
14 1
7 N
ovem
ber
2014
expectations. It was expected that the acquisition of graduate attributes would be rela-tively unrelated to the outcome variables when supervisor support was low; in thisway, the slope of the regression line in the absence of supervisor support would notbe as steep as that in the presence of supervisor support. Precisely the oppositeoccurred among subjective evaluations. Here, relatively high levels of supervisorsupport seemed to flatten out any effects of acquired graduate attributes. This suggestsa process in which supervisor support has a larger impact than graduate attributes perse. Indeed, supervisor support was also related to two other objective outcomevariables (more than graduate attributes): PhD duration and employment upon submis-sion. These data, thus, confirm prior work (e.g. Frischer and Larsson 2000; Sinclair2004) attesting to the substantive role of supervisor support during the PhD processand beyond.
An unexpected outcome of the current analyses was the extremely high internalreliability of responses to the 21 graduate attributes. An alpha coefficient of .90, ascurrently obtained, reflects strong homogeneity in responding, and it was this homo-geneity that led to the current treatment of these as a single scale instead of, say, two(e.g. Cumming et al. 2009) or four (e.g. Barrie 2004). Importantly, this can not be seensimply as an empirical anomaly. One reason for the difference between the currentsingle dimension and other models is that the items designed for the original question-naire were not developed within a formal theoretical framework but, instead, werebased upon the developers’ understandings of the extant literature as well as theirconsultation with other colleagues (Mark Western, May 7, 2010, personal communi-cation; Western et al. 2010). Indeed, inspection of the items in Table 1 reveals anabsence of items measuring Barrie’s (2006) base-level attributes; and Barrie’s level-four attributes, while possibly captured in some of the items, are likely to remainelusive to measurement in this manner. At the same time, the items do not nicelydichotomize into Cumming et al.’s categories; for example, it is not apparent whether‘a capacity for creativity’ and ‘skills in grant writing’ are academic or employabilityattributes. The discrepancy, thus, between the current findings and previous modelsserves only to emphasize the initial expressions of concern regarding the status of theliterature, and the need for seeking theoretical clarity of the meaning and nature ofgraduate attributes (see Western et al. [2010] for a different treatment of the currentitems when employing a different sub-sample of the same data set).
In addition to the findings relating to the central variables of graduate attributesand supervisor support, demographic variables remained important contributors to theoutcomes currently examined. For example, men earned higher salaries, producedmore post-PhD and had more favourable evaluations of their PhD experience.Undoubtedly, this pattern represents the broader societal status difference betweenmen and women (a pattern explored more fully with these same data by Dever et al.2008). This status difference is actually represented in the composition of the currentsample; whereas women comprise 50.3% of the Australian population (AustralianBureau of Statistics 2006), they only comprised 45.8% of the current sample. Uponfirst observing the gender differences in the regression analyses, I actually repeated allof the above analyses, but also included the interaction between supervisor supportand gender. I was particularly interested in exploring whether gender differenceswould disappear (or at least reduce) among women who reported relatively high levelsof supervisor support. If this were the case, a strong argument could be made for therole of mentorship in the pursuit of gender equality in higher education. Unfortu-nately, however, all interactions were statistically non-significant. Clearly, the
Studies in Higher Education 115
Dow
nloa
ded
by [
Nor
thea
ster
n U
nive
rsity
] at
20:
14 1
7 N
ovem
ber
2014
currently observed gender differences (and those outlined by Dever et al.) requirefurther investigation and demand practical interventions as a matter of social justice.
In a similar manner, the single effect associated with the seven Indigenous Austra-lians in the current sample (i.e. their longer time to complete their studies) alsosuggests a domain of practical intervention. Although not wanting to overstate effectsassociated with this small sub-sample, given that completion times are often used asindices for success, educators and administrators ought to take note that IndigenousAustralians are likely to be in need of additional support. Moreover, as with women,the very composition of the current sample reveals inequity between Indigenous andnon-Indigenous Australians; whereas Indigenous Australians comprise 2.6% of thetotal population (Australian Bureau of Statistics 2006), they comprised only 0.6% ofthe current sample.
The range of effects for the other demographic variables was informative and, inmany cases, makes sense. Younger students, for example, took longer to completetheir PhDs, produced more during their PhD studies, and were out of work for fewermonths after PhD submission. However, because of the lack of any systematic pattern,caution is currently taken in making any firm conclusions or interpretations. Out ofinterest, however, it is worth noting that holding a scholarship had less of an impactthan suggested by previous analyses (e.g. Higher Education Funding Council forEngland 2005; Sinclair 2004). Indeed, it may well have been expected that the absenceof a scholarship would lead to longer completion times and to less productivity duringPhD studies. The former did not occur at all, and the exact opposite of the latter didoccur. Here future research would benefit from an analysis of how scholarship statusinteracts with other variables.
As an overall word of caution, it is worth noting again that the nature of the datacurrently employed allows for only correlational inferences to be made. Moreover, thevery measures used were illustrative rather than exhaustive; clearly, inclusion of otheroutcome measures or other ways of measuring graduate attributes may well haveyielded different patterns of results. In the light of its limitations, this article should beseen as more informative than definitive. As noted in the introduction, the extant liter-ature on graduate attributes is less than clear, not the least in analyses of actual rela-tionships between attribute acquisition and subsequent outcomes. This article beginsto address this lack of clarity by quantitatively examining a relatively large sampleacross a range of outcomes.
Despite the notes of caution, the current article does report a clear and positive rela-tionship between acquired graduate attributes and productivity following PhD comple-tion. These data thus remain a testimony to the value of these attributes, and theimportant role that university education has in providing an environment for the devel-opment of attributes that can be successfully brought to bear in post-graduationemployment. At the same time, the critical role played by supervisors in producingfavourable outcomes for their students was confirmed. And, finally, potential socialinequities existing in the current Australian postgraduate education system wereobserved. Educators, administrators and students themselves are invited to learn fromthis analysis in hopes of promoting a more productive PhD experience across the board.
AcknowledgementI thank Diana Grace, Margaret Kiley, Katherine Reynolds, Malcolm Tight and three anony-mous reviewers for their helpful comments on earlier versions of this manuscript.
M.J. Platow116
Dow
nloa
ded
by [
Nor
thea
ster
n U
nive
rsity
] at
20:
14 1
7 N
ovem
ber
2014
NoteThe data presented in the current article were obtained through the Australian Social ScienceData Archive. Those who carried out the collection of the data bear no responsibility for thefurther analysis or interpretation of them.
ReferencesAiken, L.S., and S.G. West. 1991. Multiple regression: Testing and interpreting interactions.
Newbury Park, CA: Sage.Australian Bureau of Statistics. 2006. Experimental estimates of Aboriginal and Torres Strait
Islander Australians, Jun 2006. http://www.abs.gov.au/ausstats/[email protected]/Latestproducts/3238.0.55.001Main%20Features1Jun%202006?opendocument&tabname=Summary&prodno=3238.0.55.001&issue=Jun%202006#=&view= (accessed March 5, 2010).
Australian Government. n.d. Employability skills. http://www.skillsinfo.gov.au/skills/SkillsIs-sues/EmployabilitySkills/Employabilityskills.htm#Profiler (accessed March 1, 2010).
Barnett, R. 2004. Learning for an unknown future. Higher Education Research & Development23: 247–60.
Barrie, S.C. 2004. A research-based approach to generic graduate attributes policy. HigherEducation Research & Development 23: 261–75.
Barrie, S.C. 2006. Understanding what we mean by the generic attributes of graduates. HigherEducation 51: 215–41.
Berman, J.E., and T. Pitman. 2010. Occupying a ‘third space’: Research trained professionalstaff in Australian universities. Higher Education 60: 157–69.
Bills, D. 2004. Supervisors’ conceptions of research and the implications for supervisordevelopment. International Journal for Academic Development 9: 85–97.
Bridgstock, R. 2009. The graduate attributes we’ve overlooked: Enhancing graduate employ-ability through career management skills. Higher Education Research & Development 28:31–44.
Cargill, M. 2004. Transferable skills within research degrees: A collaborative genre-basedapproach to developing publication skills and its implications for research education.Teaching in Higher Education 9: 83–98.
Craswell, G. 2007. Deconstructing the skills training debate in doctoral education. HigherEducation Research & Development 26: 377–91.
Crebert, G., M. Bates, B. Bell, C.-J. Patrick, and V. Cragnolini. 2004. Developing genericskills at university, during work placement and in employment: Graduates’ perceptions.Higher Education Research & Development 23: 147–65.
Cryer, P. 1998. Transferable skills, marketability and lifelong learning: The particular case ofpostgraduate research students. Studies in Higher Education 23: 207–16.
Cumming, J., M. Kiley, M. Thomas, L. Hort, M. Pike, E. Evans, and A. Main. 2009. Researchgraduate skills project. http://www.altc.edu.au/resource-research-skill-development-questions-anu-2009 (accessed March 1, 2010).
Dever, M., W. Laffan, P. Boreham, K. Behrens, M. Haynes, M. Western, and M. Kubler.2008. Gender differences in early post-PhD employment in Australian universities: Theinfluence of the PhD experience on women’s academic careers. Brisbane: The Universityof Queensland Social Science Research Centre.
Frischer, J., and K. Larsson. 2000. Laissez-faire in research education – An inquiry into aSwedish doctoral program. Higher Education Policy 13: 131–55.
Geenan, K., P. Humphreys, and H. McIlveen. 1997. Developing transferable personal skills:Part of the graduate toolkit. Education & Training 39: 71–8.
Gilbert, R., J. Balatti, P. Turner, and H. Whitehouse. 2004. The generic skills debate inresearch higher degrees. Higher Education Research & Development 23: 375–88.
Glover, D., S. Law, and A. Youngman. 2002. Graduateness and employability: Studentperceptions of the personal outcomes of university education. Research in Post-CompulsoryEducation 7: 293–306.
Green, W., S. Hammer, and C. Star. 2009. Facing up to the challenge: Why is it so hard todevelop graduate attributes? Higher Education Research & Development 28: 17–19.
Studies in Higher Education 117
Dow
nloa
ded
by [
Nor
thea
ster
n U
nive
rsity
] at
20:
14 1
7 N
ovem
ber
2014
Griesel, H. 2003. A case study on graduate attributes. In Proceedings of the Colloquium onBuilding Relationships between Higher Education and the Private and Public Sectors, 48–58. South Africa: Council on Higher Education.
Gurr, G.M. 2001. Negotiating the ‘rackety bridge’ – A dynamic model for aligning supervi-sory style with research student development. Higher Education Research & Development20: 81–92.
Higher Education Funding Council for England (HEFCE). 2005. PhD research degrees: Entryand completion. Issues paper. London: HEFCE.
Holmes, A., and S. Miller. 2000. A case for advanced skills and employability in highereducation. Journal of Vocational Education and Training 52: 653–64.
Kiley, M., and G. Mullins. 2005. Supervisors’ conceptions of research: What are they?Scandinavian Journal of Educational Research 49: 245–62.
Manathunga, C. 2004. Developing research students’ graduate attributes. In Quality in post-graduate research: Re-imagining research education, ed. M. Kiley and G. Mullins,125–28. Adelaide, South Australia: University of Canberra.
Manathunga, C., and P. Lant. 2006. How do we ensure good PhD student outcomes?Education for Chemical Engineering 1: 72–81.
Manathunga, C., R. Pitt, and C. Critchley. 2009. Graduate attribute development and employ-ment outcomes: Tracking PhD graduates. Assessment & Evaluation in Higher Education34: 91–103.
Meyer, J.H.F., M.P. Shanahan, and R.C. Laugksch. 2005. Students’ conceptions of research. I:A qualitative and quantitative analysis. Scandinavian Journal of Educational Research 49:225–44.
Muldoon, R. 2009. Recognizing the enhancement of graduate attributes and employabilitythrough part-time work while at university. Active Learning in Higher Education 10: 237–52.
O’Brien, E.M., and S.J. Hart. 1999. Action learning: The link between academia and industry?Educational Research 41: 77–89.
Pearson, M., and A. Brew. 2002. Research training and supervision development. Studies inHigher Education 27: 135–50.
Pole, C.J., A. Sprokkereef, R.G. Burgess, and E. Lakin. 1997. Supervision of doctoral studentsin the natural sciences: Expectations and experiences. Assessment & Evaluation in HigherEducation 22: 49–63.
Sinclair, M. 2004. The pedagogy of ‘good’ PhD supervision: A national cross-disciplinaryinvestigation of PhD supervision. Canberra: Australian Government, Department ofEducation, Science and Training.
Tabachnick, B.G., and L.S. Fidell. 2007. Using multivariate statistics. 5th ed. Boston, MA:Allyn & Bacon.
Trevleaven, L., and R. Voola. 2008. Integrating the development of graduate attributesthrough constructive alignment. Journal of Marketing Education 30: 160–73.
University of Cambridge. 2009. Joint statement of the Research Councils’/AHRB’ skills train-ing requirements for research students. http://www.biomed.cam.ac.uk/gradschool/skills/skills_training.html (accessed May 31, 2010).
Western, M., P. Boreham, M. Kubler, W. Laffan, J. Western, A. Lawson, and D. Clague. 2007a.PhD graduates 5 to 7 years out: Employment outcomes, job attributes and the quality ofresearch training: Final report (revised), prepared for the Department of Education, Scienceand Training. Brisbane: The University of Queensland Social Research Centre.
Western, M., P. Boreham, M. Kubler, W. Laffan, J. Western, A. Lawson, and D. Clague.2007b. PhD graduates 5 to 7 years out: Employment outcomes, job attributes and the qual-ity of research training. Canberra: Australian Social Science Data Archive, The AustralianNational University.
Western, M., P. Boreham, A. Lawson, W. Laffan, and J. Western. 2010. Doctoral trainingand graduate outcomes: Skills and competences, graduate evaluations and job satisfac-tion. Brisbane: Institute for Social Science Research, The University of Queensland.
Wright, T., and R. Cochrane. 2000. Factors influencing successful submission of PhD theses.Studies in Higher Education 25: 181–95.
M.J. Platow118
Dow
nloa
ded
by [
Nor
thea
ster
n U
nive
rsity
] at
20:
14 1
7 N
ovem
ber
2014