This article was downloaded by: [Eindhoven Technical University]On: 15 November 2014, At: 03:06Publisher: RoutledgeInforma Ltd Registered in England and Wales Registered Number: 1072954 Registeredoffice: Mortimer House, 37-41 Mortimer Street, London W1T 3JH, UK

Journal of Further and HigherEducationPublication details, including instructions for authors andsubscription information:http://www.tandfonline.com/loi/cjfh20

Institutional Initiatives for the 1996RAEMark Baimbridge aa Department of Social and Economic Studies , University ofBradford , Richmond Road, Bradford, West Yorkshire BD7 1DP,UKPublished online: 10 Jul 2006.

To cite this article: Mark Baimbridge (1998) Institutional Initiatives for the 1996 RAE, Journal ofFurther and Higher Education, 22:2, 125-134, DOI: 10.1080/0309877980220201

To link to this article: http://dx.doi.org/10.1080/0309877980220201

PLEASE SCROLL DOWN FOR ARTICLE

Taylor & Francis makes every effort to ensure the accuracy of all the information (the“Content”) contained in the publications on our platform. However, Taylor & Francis,our agents, and our licensors make no representations or warranties whatsoeveras to the accuracy, completeness, or suitability for any purpose of the Content. Anyopinions and views expressed in this publication are the opinions and views of theauthors, and are not the views of or endorsed by Taylor & Francis. The accuracy ofthe Content should not be relied upon and should be independently verified withprimary sources of information. Taylor and Francis shall not be liable for any losses,actions, claims, proceedings, demands, costs, expenses, damages, and other liabilitieswhatsoever or howsoever caused arising directly or indirectly in connection with, inrelation to or arising out of the use of the Content.

This article may be used for research, teaching, and private study purposes. Anysubstantial or systematic reproduction, redistribution, reselling, loan, sub-licensing,systematic supply, or distribution in any form to anyone is expressly forbidden. Terms& Conditions of access and use can be found at http://www.tandfonline.com/page/terms-and-conditions

Journal of Further and Higher Education, Vol. 22, No. 2, 1998 125

Institutional Initiatives for the 1996 RAEMARK BAIMBRIDGEDepartment of Social and Economic Studies, University of Bradford, Richmond Road,Bradford, West Yorkshire BD7 1DP, UK

ABSTRACT This article reports the findings of a survey of higher education (HE)institutions encompassing a range of initiatives undertaken with respect to the 1996 researchassessment exercise (RAE). Such an analysis is pertinent given the recommendations of theDearing Report relating to research selectivity in the context of institutional funding. Thequestionnaire sought to analyse the level at which policies were formulated and the personnelconcerned with design and implementation; the reward/motivation and research 'start-up'techniques promoted; strategies concerning submissions and institutional devices to optimisesuccess; and internal funding policies. In addition to examining the response rate, thedifference between the 1992—96 institutional average weighted rating was correlated againsteach individual question enabling the statistical testing of those strategies that had a tangibleinfluence upon performance. Principal findings include: the formulation of a strategy wascrucial, although the personnel assigned to its implementation was inconsequential; themotivation and development techniques of promotion and early retirement proved especiallysuccessful; 'unfocused' submission of research output and amalgamation of staff into singleassessment categories was detrimental; the allocation of resources favoured the option of zerofunding, whilst distribution of any monies to previously highly rated departments wasinjurious.

Introduction

This article seeks to examine the various policies implemented by further education(FE) and higher education (HE) institutions participating in the fourth researchassessment exercise (RAE). It therefore aims to provide an insight for institutionalpolicy-makers into the strategies adopted by senior academics and administrativemanagers in terms of optimising research ratings and hence ultimately the securingof additional funding. The analysis is disaggregated to review the specific policiesfollowed by both 'new' and 'old' institutions since results from the both 1992 and1996 selectivity exercises indicate a division between the two strands of the formerbinary divide (Baimbridge, 1996, 1997).

Analysis of the RAEs has principally focused upon four areas (for a review seeBaimbridge, 1997). First, is the response of individual subject disciplines examiningthe associated value judgements and apparent anomalies between the contributionof research grants and publication volumes in determining rankings, the correspon-

0309-877X/98/020125-10 © NATFHE

Dow

nloa

ded

by [

Ein

dhov

en T

echn

ical

Uni

vers

ity]

at 0

3:06

15

Nov

embe

r 20

14

126 M. Baimbridge

dence between ratings and departmental size and their period of establishment. Thesecond approach sought to widen the analysis through examining the entire spec-trum of assessment units; concluding that the probability of obtaining a high ratingvaried according to an institution's historical roots, spatial location and size ofdepartments. The third theme within the existing literature focused upon publi-cations submitted for assessment in either a single or a series of leading subjectdiscipline journals, together with studies encompassing all subject categories whichconcluded that the relationship between research output and ratings differs bydiscipline. Finally, the hypothesis concerning the continuation of the binary divideas a result of the selectivity exercises has also been examined.

Thus, rather than investigating the determinants of assessment outcomes, thisstudy seeks to analyse the background policies and strategies undertaken by HEinstitutions in preparation for the 1996 selectivity exercise. In particular for the newuniversities the 1996 RAE was their first opportunity to develop specific researchorientated strategies following their initial experience of such as assessment exercisein 1992. Moreover, the recommendations of the Dealing Report suggest that suchinstitutions will be obliged to devote increased energy to their strategic planning ofRAE submissions.

The Dearing Report and Research Assessment

The examination of institutional initiatives with respect to research performance isimportant given the present framework of public funding through the 'dual support'system of the Research Councils and HE Funding Bodies. The latter providingmonies on the basis of RAE performance in addition to block grants. The DearingCommittee (NCIHE, 1997) concluded that the rigour and cost-effectiveness ofresearch has been sustained by this dual system such that the Funding Bodies andResearch Councils should remain, albeit in modified form. Thus, the current formatof four yearly RAEs will continue with the Dearing Report recommending theinclusion of international members in subject panels in the interests of transparencyand application of international standards (NCIHE, 1997; p. 186). Hence, theexamination of those initiatives which proved either beneficial or detrimental toresearch performance contained in this study will continue to be of significance forinstitutional strategists.

The Dearing Committee noted, however, that should funding to support researchfurther decline the unavoidable consequence would be for Funding Bodies toincrease the level of selectivity in the distribution of funds (NCIHE, 1997; p. 177).Whilst they would maintain resources for the leading research groups enabling themto be fully viable and competitive with their international peers, the Dearing Reportindicates that the repercussion for lesser status researchers are such that 'some partsof higher education will receive only limited amounts of research funding, or noneat all' (NCIHE, 1997; p. 177). Hence, the need for institutional policy-makers toensure the categorisation of academic staff for favourable consideration if selectivitywere to be further increased through the implementation of strategic initiatives.

The final and potentially most far-reaching implication for research performance

Dow

nloa

ded

by [

Ein

dhov

en T

echn

ical

Uni

vers

ity]

at 0

3:06

15

Nov

embe

r 20

14

Institutional Initiatives for the 1996 RAE 127

based funding is the observation within the Dearing Report that too many depart-ments are entered in the RAEs with little chance of real success. The proposedsolution is that institutions should 'be encouraged and enabled to take strategicdecisions about their role in research' (NCIHE, 1997; p. 178). This is specified toindicate that those units of assessment achieving a grade 1, 2 or 3b should in futurebe ineligible for RAE funding. Thus, the trend of exclusion is proposed to beextended to 3b departments where research equates only to national excellence. Tonegate such a financially detrimental development institutional decision-makerstherefore require detailed knowledge of those initiatives and strategies which canenhance, or detract from, research performance.

Survey Methodology

A postal survey was undertaken in the Autumn of 1996 following the RAE censusdate of March, but preceding the announcement of the findings by the subjectpanels in December. This strategy was undertaken in the expectation of maximisingreturns particularly in relation to persuading institutions to reveal their initiativesbefore outcomes were known. A total of 114 questionnaires were distributed touniversities, colleges and institutions of the HE sector throughout the UK, thus thesample constituted the membership of the Committee of Vice Chancellors andPrincipals (CVCP) together with those Colleges and Institutes of HE who enteredthe both 1992 and 1996 RAEs. The overall sample consisted of 62 old and 52 newinstitutions, with a response rate of 32 and 37 respectively, hence a total of 69completed surveys were returned for subsequent analysis [1].

The questionnaires were addressed to the Vice Chancellor, or equivalent, of eachinstitution given no prior knowledge of the personnel responsible for their RAEsubmission. Of the returned surveys, in 47 cases it was possible to identify theposition of those primarily responsible for its completion and hence by implicationthe likely institutional co-ordinators for the selectivity exercise. The two leadinggroups of respondents were senior administrators (18) followed by academic staff(17) seconded to lead the institution's research strategy, whilst the final group of 12responses emanated from the original addresses.

The survey was implicitly divided into several distinct sections whilst attemptingto encompass a plausible range of potential institutional initiatives. First, the level atwhich policies were formulated and the personnel concerned with their design andimplementation. Second, the related issues of reward/motivation techniques andresearch 'start-up' strategies institutions promoted were examined based upon thosequestions utilised in Rowley (1996). Third, the strategy undertaken concerning RAEsubmissions were analysed in relation to both the entry of staff and institutionaldevices adopted to optimise success. Finally, the survey addressed two facets offunding in terms of the sum allocated and their system of distribution. Tables I-VIIare exact representations of the questions with respondents requested to answer,where appropriate, as many categories as applicable.

The analysis of the survey is undertaken firstly through the examination of theresponses for each question. However, the primary reason for this study was to

Dow

nloa

ded

by [

Ein

dhov

en T

echn

ical

Uni

vers

ity]

at 0

3:06

15

Nov

embe

r 20

14

128 M. Baimbridge

TABLE I. At what level(s) did your institution formulate a policy/strategyconcerning the RAE?

CentralFacultyDepartmentalNone

All

682526

1

Responses

New

361191

Old

3214170

Correlation

All

+ ve**n/s

+ ve**-ve**

New

n/sn/sn/sn/s

Old

n/sn/sn/sn/a

***=p<0.01, **=/><0.05 and*=p<0.1 .

analyse the consequences of adopting various initiatives concerning the RAE.Therefore the difference between the 1996 and 1992 average weighted rating [2] foreach institution (THES, 1992, 1996) was calculated and then correlated againsteach individual question for all, new and old institutions. Such a procedure enablesthe empirical testing of which strategies had a tangible influence upon performancein the 1996 selectivity exercise relative to the outcome in the previous RAE. Thisstatistical analysis is key to determining which policies proved crucial since it mustbe remembered that merely because either the majority or minority of institutionsundertook specific initiatives they cannot be pre-judged as being necessarily 'effec-tive' or 'ineffective'. The positive and negative signs indicate the directional impactof statistically significant factors, with those insignificant denoted by the term 'n/s'.Where a correlation coefficient could not be calculated due to no response for aparticular question the term 'n/a' is entered. The accompanying asterisks indicatethe level, or probability, of the statistical significance, hence: ***=/><0.01,**=p<0.05 and *=p<0.l.

Results and Discussion

Tables I and II present the survey findings in terms of the level of policy formulationand the personnel ensured with its design and implementation. The largest numberof respondents chose a centrally constituted policy towards the 1996 RAE with onlyone failing to specifically formulate any strategy. The correlation statistics indicatethat for all institutions a positive relationship exists between both central anddepartmental levels of policy formulation and the change in RAE performance.However, the failure to develop any policy appears detrimental, being negativelycorrelated to the difference between assessment exercises. In contrast, none of thecorrelation coefficients are found to be statistically significant for question concern-ing the design and implementation of the institution's policy (Table II), althoughthe re-assignment of exiting personnel was evidently the preferred choice for themajority of new and old universities.

With respect to the second category of questions concerning reward/motivationtechniques and 'start-up' strategies Tables III and IV summarise their findings. Theoverall highest response rate is found for the reward option of 'reduction in teaching

Dow

nloa

ded

by [

Ein

dhov

en T

echn

ical

Uni

vers

ity]

at 0

3:06

15

Nov

embe

r 20

14

Institutional Initiatives for the 1996 RAE 129

TABLE II. In designing and implementing this policy/strategy did you

Responses Correlation

Re-assign existing personnelfrom your institution

Newly/specificallyappointed personnel

Employ consultants

All

67

6

2

New

36

5

1

Old

31

2

1

All

n/s

n/s

n/s

New

n/s

n/s

n/s

Old

n/s

n/s

n/s

loads' closely followed by internal peer respect and specifically allocated researchtime (sabbaticals, etc). The former and latter of these are mirrored for newinstitutions where the 'opportunity for travel/conference attendance' was also apopular reward/motivation technique. However, for the old universities the prospectof 'promotion' was a leading strategy. The correlation figures indicate that for theentire sample the motivation of promotion exerted a significant positive influence onperformance, whilst the 'opportunity for travel/conference attendance' was mar-ginally detrimental. However, the creation of an 'elite' group of staff who may be ona 'fast track' of promotion or general career advancement through greater emphasisbeing placed on research, to the possible detriment of teaching, is likely to proveproblematic to collegiate harmony (Baimbridge, 1996). For new institutions, bothinternal and external recognition coincided with improved ratings whilst the policyof no rewards had a negative impact, although such a strategy within old universitiesraised research ratings. This paradoxical result could result from the likelihood thatmany of these latter institutions have implement such reward/motivation techniques

TABLE III. Which reward/motivation technique(s) did your institution implement?

PromotionPerformance related payInternal peer respectExternal peer recognitionOpportunity for travel/

conference attendanceReduction in teaching loadsReduction of administration tasksSpecifically allocated research

time (sabbaticals, etc.)None

All

247

372633

382836

14

Responses

New

72

191421

221520

8

Old

175

181212

161316

6

All

+ ve**n/sn/sn/s

— ve*

n/sn/sn/s

n/s

Correlation

New

n/sn/s

+ ve*+ ve*n/s

n/sn/sn/s

-ve**

Old

n/sn/sn/sn/sn/s

n/sn/sn/s

+ ve**

***=p<0.01,**=p<0.05 and*=p<0.1.

Dow

nloa

ded

by [

Ein

dhov

en T

echn

ical

Uni

vers

ity]

at 0

3:06

15

Nov

embe

r 20

14

130 M. Baimbridge

TABLE IV. Which research development/'start-up' strategies did your institution promote?

Internal research seminarsPublications workshopsResearch newsletterTeam researchConference attendanceInternal publication seriesProfessional journal articlesAppointment of new more

research orientated staffAppointment of readers/

professors as research leadersInitiate programme of

early retirement

All

4617202942212748

47

19

Responses

New

3111121426111430

28

6

Old

1568

1516101318

19

13

All

-ve***n/sn/sn/s

-ve**n/sn/s

-ve**

n/s

+ ve***

Correlation

New

n/sn/sn/sn/sn/sn/sn/s

+ ve*

n/s

+ ve***

Old

n/sn/sn/sn/sn/sn/sn/sn/s

n/s

n/s

***=p<0.01, **=p<0.05 and*=/><0.1.

for some time given that the questionnaire was specifically directed to initiativesestablished following the 1992 RAE.

Table IV presents the results relating to various development or 'start-up'strategies promoted in the period leading up to the 1996 assessment exercise. Thethree categories of 'appointment of new more research orientated staff, 'appoint-ment of Readers/Professors as research leaders' and 'internal research seminars'encompass the leading responses. However, the correlation analysis indicates thatoverall the former and latter of these policies together with 'conference attendance'were inversely associated with the difference in RAE ratings. Although the 'appoint-ment of new more research orientated staff in relation to new institutions displaysthe expected positive relationship to research performance, the only other questionto prove statistically significant concerned the initiation of a 'programme of earlyretirement' which is also positively correlated for the total sample. Thus, eventhough it is amongst the lowest response rates this development strategy is stronglyassociated with those HE establishments witnessing improved performance,appearing to reinforce the notion that the 'conversion' of staff into active researchersremains problematic (Baimbridge, 1997). Hence, a research development strategywhich could surmount this difficulty in the guise of an early retirement programme.However, this is frequently regarded as a potentially hazardous area of employee-relations since not only does it raise issues of what precisely constitutes being'research-active', but also ignores that so-called 'teaching-only' staff undertake therevision of teaching and learning materials which can be regarded as a form ofresearch activity (Rowley, 1996).

The third section of the questionnaire sought to examine the submission policy interms of the quantity and quality of research output together with institutionalstrategies to optimise performance. With respect to the former issue, Table V

Dow

nloa

ded

by [

Ein

dhov

en T

echn

ical

Uni

vers

ity]

at 0

3:06

15

Nov

embe

r 20

14

Institutional Initiatives for the 1996 RAE 131

TABLE V. Which strategies did your institution implement concerning its RAE submissions?

Enter all staff withany publications

Enter all staff with aminimum of four publications

Enter staff with a minimumof articles and/or books

Enter staff with a minimum ofonly refereed journal articles

Combine staff across departmentsto fit RAE assessment categories

Enter all non-publishing staffinto 'dummy' assessment categories

All

2

28

13

10

31

7

Responses

New

0

16

10

7

19

5

Old

2

12

3

3

12

2

Correlation

All

n/s

n/s

— ve**

n/s

- v e *

n/s

New

n/a

— ve*

n/s

n/s

n/s

n/s

Old

n/s

n/s

n/s

n/s

n/s

n/s

***=p<0.01, **=p<0.05 and *=/><0.1.

indicates that the fundamental requirement of submitting 'staff with a minimum offour publications' was pursued by the largest number of respondents. However, thispolicy together with entering 'all staff with a minimum of articles and/or books' fornew institutions, is inversely related to the resulting difference in RAE ratings. Thissuggests that such relatively 'unfocused' strategies whilst likely to raise the pro-portion of staff submitted are ultimately detrimental as subject panels appear toplace greater emphasis upon quality rather than the number of entries in anindividual assessment unit. Although in relation to general polices for submissionsthe combining of 'staff across departments to fit RAE assessment categories' wasundertaken by nearly half of the total sample, the correlation is marginally inverselysignificant for such a strategy suggesting that a degree of departmental reorgani-sation potentially remains to be undertaken.

The final category of questions addressed funding initiatives for the 1996 assess-ment exercise (Tables VI and VII). The overall sample results show that the twohighest response rates were for the diametrically opposite levels of '£500 000-£l m'and 'none'. This finding is also reflected in the breakdown of new and oldinstitutions, where if funding was allocated at all, it was clustered in the range of£200 000 to £1 m for the former and at 'more than £1.5 m' for the latter institu-tions. The surprising result, however, from the correlation analysis is that only theallocation of no monies is positively related to RAE performance, whilst large, smalland intermediate sums are shown to be detrimental to significant ratings improve-ment. Hence, the correlation testing reveals relationships between funding moniesand performance not immediately evident from examination of the response rate. Inparticular the notion of institutions being able to 'buy' research success, for instancethrough the importation of established staff, appears to be questionable based uponthese survey findings.

Finally, Table VII examines the influence of performance in the preceding

Dow

nloa

ded

by [

Ein

dhov

en T

echn

ical

Uni

vers

ity]

at 0

3:06

15

Nov

embe

r 20

14

132 M. Baimbridge

TABLE VI. How much funding did your institution allocate to raise performancein the RAE?

NoneLess than £50,000£50,000-£100,000£100,000-£200,000£200,000-£500,000£500,000-£l m£lm-£1.5mMore than £1.5 m

All

104548

1249

*** =p<0.0l, ** =/><0.05

Responses

New Old

5 52 23 22 27 18 43 13 6

and*=p<0.1 .

TABLE VII. What type of system(s) were these

Competitive bid systemEqually to all departmentsEqually to all facultiesIn favour of the best performing

departments in the 1992In favour of the poorest

performing departmentsthe 1992 RAE

RAE

in

Response

All New

25 113 24 3

30 22

0 0

All

n/sn/sn/sn/s

— ven/sn/sn/s

monies

Old

14118

0

Correlation

New

n/sn/sn/sn/s

* n/sn/sn/sn/s

distributed

Old

+ ve**- v e * *

n/sn/sn/sn/s

- v e *n/s

through

Correlation

All

n/sn/sn/s

— ve***

n/a

New

n/sn/sn/sn/s

n/a

Old

n/sn/sn/sn/s

n/a

***=£<0.01, **=/><0.05 and*=p<0.1 .

selectivity exercise upon the distribution of funding. The largest response was infavour of a 'competitive bid system' with the two egalitarian approaches provingunpopular, although none of these alternatives were found to significantly impactupon differences in the average weighted rating. Although the dominant distributionstrategy was for monies 'in favour of the best performing departments in the 1992RAE', this is inversely associated with improved performance levels. Such a findingappears to confirm the hypothesis that rewarding high performing departments isfrequently the reverse of good sense, as gains are more likely to be forthcoming bydirecting resources to relatively under-performing departments rather than thosenear their productivity frontier (Baimbridge, 1996, 1997).

Conclusion

This article has sought to offer an insight into the various institutional initiativesundertaken for the 1996 RAE. The study has potentially important implications for

Dow

nloa

ded

by [

Ein

dhov

en T

echn

ical

Uni

vers

ity]

at 0

3:06

15

Nov

embe

r 20

14

Institutional Initiatives for the 1996 RAE 133

the implementation of successful strategies in the enhancement of research per-formance by institutional administrators and policy-makers. Not only is the incomefrom such exercises frequently crucial in terms of financial viability, but beneficialexternalities are also likely. For instance, the retention and recruitment of highcalibre staff, external funding opportunities, undergraduate and postgraduate appli-cations, and the ethereal notion of institutional prestige with possible consequencesfor staff morale.

Although this has only been an initial examination of institutional practices, anumber of key findings emerge from the study. First, the formulation of a strategywas crucial whether at central or departmental levels, although the personnelassigned with its implementation proved inconsequential. Second, whilst severalmotivation and development techniques were of marginal benefit or detriment,those of promotion and early retirement proved especially successful, if not withoutprofound repercussions. Third, a relatively unfocused submission policy of researchoutput and the amalgamation of diasporic staff into single assessment categories wasseen to be detrimental. Finally, the allocation of specific resources to enhance RAEperformance favoured the option of zero funding, whilst the distribution of anymonies to previously highly rated departments was injurious.

The results of this study therefore raise potentially important issues concerningthe introduction and long-term pursuit of institutional initiatives relating to researchoutput as measured by the past RAEs. Thus, further examination of the detaileddevelopment and implementation of such strategies is indicated to investigate theprecise nature of these apparent relationships. Indeed, following the recom-mendations of the Dearing Committee in relation to research selectivity, theincreased likelihood of strategic decision-making by institutions will become acentral feature of much debate within the UK academic community.

Notes[1] With a sample size of n>30 this analysis therefore negates a frequent difficulty of such

questionnaire based studies in being confined to 'small' sample sizes which can proveproblematic for the reliability of results.

[2] The average weighted rating takes into account not only the ratings given in the researchselectivity exercise, but also the size of departments, with each individual rating multipliedby the number of submitted full-time equivalent research staff in the department beingassessed. The totals for each institution are summed and then divided by the total numberof active research staff to arrive at the average (THES, 1996).

ReferencesBAIMBRIDGE, M. (1996) The return of the binary divide? Evidence from the 1992 RAE, Journal

of Further and Higher Education, 20(1), pp. 3-13BAIMBRIDGE, M. (1997) Institutional research performance 1992-1996: a tale of two sectors,

Journal of Further and Higher Education, 21(1), pp. 69-78.NATIONAL COMMITTEE OF INQUIRY INTO HIGHER EDUCATION (NCIHE) (1997) Higher education

in the learning society—main report, (The Dearing Report); (London, DFEE).ROWLEY, J. (1996) Making the tension between research and teaching creative in business and

management: a pilot study, Journal of Further and Higher Education, 20(1), pp. 74-92.

Dow

nloa

ded

by [

Ein

dhov

en T

echn

ical

Uni

vers

ity]

at 0

3:06

15

Nov

embe

r 20

14

134 M. Baimbridge

TIMES HIGHER EDUCATION SUPPLEMENT (THES) (1992) League table of excellence, 18December, p. ii.

TIMES HIGHER EDUCATION SUPPLEMENT (THES) (1996) League table of excellence, 20December, p. xvi.

Dow

nloa

ded

by [

Ein

dhov

en T

echn

ical

Uni

vers

ity]

at 0

3:06

15

Nov

embe

r 20

14