DATA ENVELOPMENT ANALYSIS AND ITS APPLICATION TO THE

MEASUREMENT OF EFFICIENCY IN HIGHER EDUCATION

Jill Johnes*

Department of Economics

Lancaster University Management School

Lancaster University

Lancaster LA1 4YX

United Kingdom

e-mail: j.johnes@lancaster.ac.uk

Final Version: February 2005

*I am grateful to two anonymous referees for their comments. All errors are mine.

1

ABSTRACT

The purpose of this paper is to examine the possibility of measuring efficiency in the

context of higher education. The paper begins by exploring the advantages and

drawbacks of the various methods for measuring efficiency in the higher education

context. The ease with which data envelopment analysis (DEA) can handle multiple

inputs and multiple outputs makes it an attractive choice of technique for measuring

the efficiency of higher education institutions (HEIs), yet its drawbacks cannot be

ignored. Thus a number of extensions to the methodology, designed to overcome

some of the disadvantages, are presented. The paper ends with an application of DEA

to a data set of more than 100 HEIs in England using data for the year 2000/01.

Technical and scale efficiency in the English higher education sector appear to be

high on average. The Pastor et al (2002) test for comparing nested DEA models is

useful in reducing the full model to a smaller 'significant' set of inputs and outputs.

Thus the quantity and quality of undergraduates, the quantity of postgraduates,

expenditure on administration, and the value of interest payments and depreciation

are significant inputs to, and the quantity and quality of undergraduate degrees, the

quantity of postgraduate degrees and research are significant outputs in the English

higher education production process. The possibility of differences in the production

frontier (and hence the distribution of efficiencies) of three distinct groups of HEIs is

explored using a test proposed by Charnes et al (1981), but no significant differences

are found. Bootstrapping procedures, however, suggest that differences between the

most and least efficient English HEIs are significant.

JEL Classification: I21, C14

Keywords: efficiency measurement, data envelopment analysis, higher education

2

1. INTRODUCTION

The higher education sectors of many countries obtain at least some of their

income from public funds making it essential, in the interests of accountability, to

measure the efficiency of the institutions which comprise these sectors. The higher

education sector, however, has characteristics which make it difficult to measure

efficiency: it is non-profit making; there is an absence of output and input prices; and

higher education institutions (HEIs) produce multiple outputs from multiple inputs.

An assortment of methodological approaches have been employed in an effort to

resolve the problem of efficiency measurement in this context, from early studies

which use ordinary least squares (OLS) regression methods (Johnes & Taylor 1990),

to more recent studies which use frontier methods such as data envelopment analysis

(DEA) (Athanassopoulos & Shale 1997; Räty 2002; Abbot & Doucouliagos 2003;

Johnes, forthcoming) or stochastic frontier analysis (SFA) (Izadi et al 2002).

The purpose of this paper is to explore the issue of the measurement of

technical efficiency in the context of higher education1. From an output-oriented

perspective (Farrell 1957), efficiency is defined as the ratio of a firm's observed

output to the maximum output which could be achieved given its input levels. Since a

firm's observed production point is known, the measurement of efficiency therefore

requires the estimation of the production function in order to estimate its potential

production point. Various parametric and non-parametric techniques of estimation can

be used.

The paper is in five sections of which this is the first. Section 2 provides a

brief overview of methods for estimating the higher education production function 1 The choice of technical as opposed to alternative types of efficiency is made on the basis of the variables in the data set which will be used in the empirical analysis. Alternative measures of efficiency include overall (economic) efficiency and social efficiency. These types of efficiency have been examined in the context of higher education (see, for example, Athanassopoulos & Shale 1997; Korhonen et al 2001; Abbot & Doucouliagos 2003; Izadi et al 2002).

3

and provides a summary of the advantages and drawbacks of the various methods in

the context of HEIs. The non-parametric method of DEA is presented in detail in

section 3, along with developments and extensions of DEA designed to overcome its

main drawbacks. The results of applying DEA to a data set of more than 100 English

universities are presented in section 4. In particular, various DEA models are

compared using the Pastor et al (2002) test, possible differences between subgroups in

terms of the distribution of efficiencies are investigated using a method suggested by

Charnes et al (1981), and bootstrapping procedures (Simar & Wilson 1998; 1999) are

applied to produce confidence intervals for the efficiency scores. Conclusions

regarding the efficiency of the English higher education sector and the usefulness of

DEA and its extensions as a technique for measuring efficiency in this context are

drawn in section 5.

2. ESTIMATING THE HIGHER EDUCATION PRODUCTION FUNCTION

There are two basic approaches to estimating a production function: the

statistical (or econometric) approach and the non-statistical (or programming

approach). Under the statistical approach, the production function can be represented

by

kumkkk exxfy −= ),...,( 1 (1)

where yk is the output of producer k; xik is the amount of the ith input (i = 1, …

m) used by producer k; 0≥ku and uk represents the inefficiency of producer k (Lovell

1993), and a specific distribution is assumed for the uk (Førsund et al 1980).

Technical efficiency of firm k (TEk) is then measured by

ku

mkk

kk e

xxfy

TE −==),...( 1

(2)

4

The statistical approach is often parametric since a particular functional form

for the production function is also assumed. Equation 1 and hence the measures of

inefficiency (uk) can be estimated using a variety of statistical techniques including

corrected ordinary least squares, modified ordinary least squares and maximum

likelihood estimation (Lovell 1993)2. While these methods provide estimates of the

parameters of the frontier, the significance of which can be tested, they are beset by

the problem of possible misspecification. In addition, they are not easily applied in a

situation where there are multiple inputs and multiple outputs, a serious drawback in

the context of higher education.

DEA is a non-statistical and non-parametric approach which makes no

assumptions regarding the distribution of inefficiencies or the functional form of the

production function (although it does impose some technical restrictions such as

monotonicity and convexity – see Färe et al 1994). Instead, it uses the input and

output data themselves to compute, using linear programming methods, the

production possibility frontier. The efficiency of each unit is measured as the ratio of

weighted output to weighted input, where the weights used are not assigned a priori,

but are calculated by the technique itself so as to reflect the unit at its most efficient

relative to all others in the dataset. In a multi-output, multi-input production context,

DEA provides estimates of the distance function (Shephard 1970), which is a

generalization of the single output production function. The advantages of the

distance function approach are, first, that there is no need to make behavioural

assumptions about the firms, such as cost minimisation or profit maximisation (which

would be regarded as inappropriate in the higher education context), and, second,

knowledge of input and output prices, which are often unknown in the higher 2 Note that OLS estimates a production function which is an average of the production points rather than an envelope around them, and so it would not be an appropriate estimation method as the inefficiencies would not be constrained to be positive.

5

education context, is not required. The lack of assumptions in DEA regarding

statistical distributions, however, means that there are no estimates or significance

tests of the parameters of the production function, a potentially serious problem if

results are sensitive to the specification of inputs and outputs.

The methods considered so far have all assumed that deviations from the

production function are deterministic and hence are a consequence solely of

inefficiency. Under a stochastic approach such as stochastic frontier analysis (SFA),

however, the residual is separated into two components: one the result of inefficiency

and one random. In practice, this involves assuming a specific distribution for each

error component. Thus the SFA production function can be written as

kmkkk exxfy ε),...( 1= (3)

where kkk uv −=ε , ( )2,0~ vk Nv σ , uk and vk are statistically independent

and 0≥ku (Aigner et al 1977). One component of the residuals (vk) is normal and is

attributed to measurement error and random fluctuations, while the second component

(uk) is one-sided (typically exponential or half-normal) and is attributed to technical

inefficiency. A stochastic approach therefore produces efficiency measures which are

separated from random shocks or measurement errors, but they are still potentially

affected by misspecification errors. The imposition of a particular distributional form

(eg. half-normal or exponential) on that component of the residual which is attributed

to technical inefficiency is an assumption which has no theoretical basis. In addition,

SFA is not easily applied in a multiple input multiple output production situation.

The analyst is therefore faced with an array of methods for estimating the

higher education production function and deriving measures of efficiency. The

multiple input multiple output nature of production in higher education combined

with the absence of prices (of both inputs and outputs) make DEA an attractive choice

6

of methodology in this context, despite its shortcomings. The next section therefore

presents the DEA technique and investigates how some of its drawbacks are

overcome by extensions to the technique.

3. THE DEA METHODOLOGY

3.1 An overview: DEA was developed by Charnes et al (1978) following work

by Dantzig (1951) and Farrell (1957), and estimates a piece-wise linear production

function relative to which the efficiency of each firm, or decision making unit (DMU)

can be measured. In its simplest form, DEA assumes constant returns to scale (CRS).

Consider, a simple example of 5 universities (A, B, C, D, E) producing 2 outputs, y1

(for example, the number of graduates achieving 'good' degrees) and y2 (for example,

the number of graduates going into employment) using the input x (for example, the

number of undergraduates). Figure 1 plots the ratio of output y1 to x against the ratio

of output y2 to x, and the piecewise linear boundary which joins up universities A, B,

C and D is the production frontier. All DMUs on the frontier are efficient since none

can produce more of both outputs (for a given input level) than any other unit on the

frontier. In contrast, university E, which lies inside the frontier is inefficient, and the

ratio 'OEOE measures university E's efficiency relative to the other DMUs in the

data set.

<Figure 1 here>

7

3.2 DEA under variable returns to scale (VRS): The CRS assumption can be

relaxed and the DEA model can be easily modified to incorporate variable returns to

scale (VRS) (Banker et al, 1984). While choice of orientation does not affect

efficiencies under CRS, it does under the assumption of VRS (Coelli et al 1998),

although it has been shown only to have a slight influence in many cases (Coelli &

Perelman 1999). In an input orientation, outputs are assumed to be fixed and the

possibility of proportional reduction in inputs is explored, whereas, in an output

orientation, it is inputs which are fixed while the possibility of a proportional

expansion of outputs is explored. The latter orientation is deemed the more

appropriate in this study where the quantity and quality of the inputs, such as student

entrants, are fixed.

In an output-oriented framework and under the assumption of VRS, the

following linear programming model needs to be solved for each DMU in the data set

in order to calculate DEA efficiencies.

Maximize ∑∑==

++m

ii

s

rrk ss

11εεφ (3)

Subject to 01

=+−∑=

r

n

jrjjrkk syy λφ sr ,...,1= (4)

01

=−−∑=

i

n

rijjik sxx λ mi ,...,1= (5)

∑=

=n

jj

11λ (6)

0,, ≥irj ssλ misrnj ,...,1;,...,1;,...,1 ===∀

where there are s outputs and m inputs; yrk is the amount of output r used by

DMU k; xik is the amount of input i used by DMU k; and ir ss , are the output and

8

input slacks respectively. Technical efficiency of DMU k is measured by kφ

1 ; DMU

k is efficient if its efficiency score is 1 and all slacks are zero. The VRS dual differs

from the constant returns to scale (CRS) dual only by the inclusion of the constraint in

equation (6). Comparison of the efficiencies derived from the above with the CRS

efficiencies allows the derivation of measures of pure technical efficiency and scale

efficiency.

3.3 Issues in the specification of inputs and outputs: Two issues need to be

considered in the context of specification. The first relates to the initial measurement

and specification of the input output set, and the second to the importance of each of

the inputs and outputs in the DEA model. There are considerable problems of defining

and measuring the inputs and outputs of the higher education production process

(Johnes 2004). Further concerns arise from the distinction between inputs which can

be controlled by the HEIs under investigation, and those which cannot, such as

environmental factors. There are two contrasting approaches to the problem. The first

approach is to include all inputs, whether controllable or not, in the efficiency analysis

(Cubbin & Tzanidakis 1998; Grosskopf 1996). This is generally the approach which

has been taken in DEAs applied to the higher education sector, but can produce results

which do not make adequate allowance for HEIs facing a harsh environment, and

their inefficiency may be overestimated as a consequence3.

The second approach is to adopt a two-stage procedure whereby the efficiency

scores for a set of institutions are derived using DEA and including a subset of

controllable inputs, and then these efficiencies are analysed at a second stage in

relation to the non-controllable inputs using an appropriate transformation and

3 This problem is addressed by Ruggiero (1996) who develops an approach, in the context of DEA, so that each DMU has in its reference set only those DMUs which face at least as harsh an environment as itself.

9

statistical technique. The theoretical difference between the two approaches is that a

two-stage procedure assumes that the second stage input variables affect the efficiency

with which the outputs are produced from the inputs, whereas the one-stage procedure

assumes that all the inputs affect the process of production of the outputs from the

inputs (Lovell 1993). In practice, it can be difficult to distinguish between the inputs

which should be included at the first stage, and those which should be included in the

second stage. Additional problems with the two-stage procedure include, first, the

possible introduction of misspecification errors at the second stage, and, second, that

the DEA efficiency estimates are serially correlated making standard methods of

inference invalid (Simar & Wilson 2004a). Results from comparisons made in the

context of secondary education suggest that there is little difference between the

efficiencies derived from a two-stage approach and those from a one-stage DEA

(McCarty and Yaisawarng 1993).

A serious drawback of DEA is that it does not provide tests of the significance

of the input or output variables included in the model. Empirical studies have largely

dealt with this by performing DEA on a variety of specifications to check the

sensitivity of results. More recently, Pastor et al (2002) have developed a test

(analogous to an F test on a subset of variables in a multiple regression) for assessing

the significance of nested models in a radial DEA (see appendix 1). This test has not,

to date, been applied in the context of efficiency in higher education, but examples of

applications can be found in Lovell & Pastor (1997), Mancebon & Bandrés (1999),

and Mancebon and Mar Molinero (2000).

3.4 Comparing the efficiency of subgroups of DMUs: If there are differences in

efficiency between specific subgroups of the full sample (for example, public versus

private institutions in the USA), it is more appropriate to apply DEA separately to

10

each subgroup in order to derive appropriate peer groups for the inefficient DMUs.

One method (Charnes et al 1981) for checking for differences involves applying DEA

to the subgroups (1 and 2, say), and then projecting all inefficient observations on to

their own efficiency frontier (or 'α -envelope' for each α = 1 and α =2). The DEA is

run again on the data of projected and efficient DMUs, pooled across both subgroups,

in order to derive an 'inter-envelope'. The efficiency scores from this last DEA can

then be used to test, using a suitable non-parametric test, whether there are significant

differences between the efficiency distributions for each group.

3.5 Confidence intervals for efficiency scores: One of the attractions of DEA is

that it provides a simple score of efficiency for each firm, understood by everyone,

even though the production process itself may be highly complex. Although DMUs

may appear to vary widely in their efficiency (as denoted by the DEA efficiency

score), the basic DEA technique provides no indication whether the difference

between DMUs is statistically significant. The development of bootstrapping

procedures (Simar & Wilson 1998; 1999; 2004b) allow us to estimate 95% confidence

intervals for each HEI's efficiency score (see appendix 2) and these can be used to

investigate whether the efficiencies derived differ significantly between universities.

4. AN APPLICATION OF DEA TO HEIs IN ENGLAND

4.1 Data and methodology: Data collected on inputs and outputs for universities

in England for the academic year 2000/01 form the basis of the analysis. The number

of first degree graduates weighted by their degree classification4 (GRADQUAL) is

included to capture both the quantity and quality of undergraduate teaching output.

The total number of graduates from higher degrees (POSTGRAD) is included to

reflect the quantity of postgraduate output (as in Athanassopoulos & Shale 1997). The 4 The weights used in the results presented here are first = 30; upper second = 25; lower second = 20; third = 15 and unclassified = 10. See table 1 for precise definition of GRADQUAL. Various alternative weights were applied and had no effect on the results or conclusions.

11

grant for research provided by the Higher Education Funding Council for England

(HEFCE) (which reflects the Research Assessment Exercise quality rating and the

number of research active staff and is therefore similar to the Quantum Research

measure used by Avkiran 2001 in his study of Australian universities) is included to

reflect both the quality and quantity of research output (RESEARCH).

The quantity and quality of undergraduate inputs are captured by including a

composite measure (UGQUAL) which is the product of the number of undergraduates

and the average A level score of undergraduate entrants (thus matching the composite

measure of the quantity and quality of undergraduate teaching output, GRADQUAL).

This differs from the approach adopted by Athanassopoulos & Shale (1997), who

include average A level score of undergraduate entrants and undergraduate numbers

as two separate variables, but has the advantage that the undergraduate input and

output measures are comparable in measuring both quantity and quality. The total

number of postgraduates (PG) is included to reflect quantity of postgraduate input

while the number of teaching and research staff (STAFF) measures the staff input to

the higher education production process.

Three variables are included to reflect additional inputs to the higher education

process: expenditure on administration, expenditure on library and computer facilities,

and the value of interest payments and depreciation (denoted by, respectively,

ADMIN, LIBCOMP and CAPITAL). Precise definitions of the data and their sources

can be found in table 1.

<Table 1 here>

In total, data are available on all variables for 109 English HEIs. The analysis

performed here differs from previous studies of technical efficiency in UK

universities (for example, Athanassopulos & Shale 1997) because it is based on a

12

sample of data which includes pre-1992, post-1992 and Standing Conference of

Principals Ltd (SCOP) HEIs. HEIs in England can be divided into three groups on the

basis of their historical background: pre-1992 universities, post-1992 universities and

SCOP colleges. The pre-1992 universities had the status of a university before the

provisions of the Further and Higher Education Act of 1992 came into force. Prior to

1992, they were largely funded by the Universities Funding Council. The post-1992

universities are mostly former polytechnics which, prior to 1992, were funded by the

Polytechnics and Colleges Funding Councils. The Further and Higher Education Act

of 1992 allowed these HEIs to award their own degrees and to use the title of

university. The SCOP colleges are part of the unified higher education sector of

England, but differ from other HEIs in that they are often specialist institutions

concentrating on a particular discipline such as music, drama, performing arts,

education or agriculture. In fact, for the purposes of this analysis, the third group

includes all SCOP and SCOP-type HEIs (i.e. HEIs which are not officially SCOPs but

which have similar characteristics). This data set therefore provides the opportunity to

establish whether there are differences between the three types of HEIs in terms of

efficiency of output production.

Descriptive statistics for all input and output variables are displayed in table 2

for all HEIs together and for each subgroup of HEIs. It is clear from table 2 that

research and postgraduate outputs are more highly concentrated in the pre-1992

universities and least concentrated in the SCOP colleges, and undergraduate teaching

output is most concentrated in the post-1992 HEIs. This is balanced, however, by

large differences in inputs. SCOP colleges, in particular, have considerably smaller

quantities of all inputs than the pre- and post-1992 HEIs.

<Table 2 here>

13

The package Warwick DEA is used to run a DEA performed on the assumption

that all the defined inputs affect the process of production (i.e. a one-stage procedure)

using an output-oriented approach. A VRS model is used in the first instance, and

scale efficiency is examined subsequently.

4.2 Results: Initially, DEA is applied to the full data set of 3 outputs and 6

inputs (see table 1), and the Pastor et al (2002) test, with values 1.1=ρ and

15.00 =p , is used to assess the significance of individual variables and of groups of

variables. The removal of STAFF and LIBCOMP results in a change in efficiency

scores which is not substantial (the Pastor et al p-value = 0.99). Moreover, the rank

correlation between the efficiency scores of this reduced model and the full model is

highly significant (Spearman's rank correlation coefficient = 0.92). The possibility of

further removal of variables is investigated by removing one variable at a time from

the model. The resulting Pastor et al p-values are close to zero, suggesting that that

the contribution to the model of each variable is relevant (see models M1 to M8 in

table 3 for details of these results). Thus the model containing 3 outputs

(GRADQUAL, POSTGRAD and RESEARCH) and 4 inputs (UGQUAL, PG ADMIN

and CAPITAL) is the preferred model using the Pastor et al test as the preference

criterion.

<Table 3 here>

The deletion of STAFF which would be considered a priori to be a crucial input

to the production process requires further investigation and discussion. An

examination of the inputs reveals that STAFF is highly significantly correlated with

14

the inputs which remain in the preferred model5, and this possibly explains the lack of

significance of this variable.

Table 4 displays the results of the two specifications (reduced and full) by

subgroup. The general efficiency across all English universities is very high: the

average level of efficiency varies from 93% to 95% across the two models, and the

number of efficient DMUs varies from 51 to 61. In contrast to this broad picture of a

highly efficient higher education sector, some individual HEIs have efficiency scores

which are considerably lower than the mean, the lowest score being around 60%.

These findings of the efficiency of the higher education sector are broadly in line with

findings derived using DEA on an earlier sample of UK universities (Athanassopoulos

& Shale 1997), and with findings from the Australian higher education sector

(Avkiran 2001; Abbot & Doucouliagos 2003). Comparisons with the results of DEA

applied to hospitals (a similar non-profit context) suggest that the results are similar in

mean efficiency, but that both the proportion of efficient DMUs and the range of

efficiency are lower in the higher education context (Byrnes & Valdmanis 1994).

Comparisons of the results with those derived from sectors with profit motivation are

mixed. For example, results from assessing bank branch performance give a similar

mean efficiency score and proportion of technically efficient DMUs (Paradi et al

2004), while results from an assessment of the efficiency of Norwegian ferries suggest

a much lower level of efficiency (both in mean and the proportion of technically

efficient DMUs) (Førsund & Hemaes 1994).

<Table 4 here>

The apparent high level of efficiency in this and other studies of the efficiency

of the higher education sector warrants further discussion given that this is a sector

5 Pearson’s correlation coefficient between STAFF and, respectively, UGQUAL, PG, ADMIN and CAPITAL is 0.886, 0.844, 0.836 and 0.861.

15

where there is no profit motivation. One possible explanation of this result is that

DEA produces a measure of efficiency relative to that achieved by the other DMUs in

the study. Thus, the production frontier estimated by DEA may not in fact be the true

frontier which could be achieved if the sector were truly efficient; it is merely the

observed production frontier for the sector. If this is the case, then overall levels of

efficiency are overestimated by DEA, but rankings of and comparisons between the

DMUs are likely still to be valid. Another explanation is that, while the English higher

education sector has no profit motivation, it has been increasingly exposed to market

forces over the last decade. HEIs must compete against each other to attract the best

students and funds for research, thus providing incentives for efficiency.

It is possible from table 4 to examine the efficiency scores in the context of the

different subgroups. While the mean efficiency scores suggest that technical

efficiency is highest, on average, amongst pre-1992 universities, and lowest amongst

the SCOP and SCOP-type colleges, an F-test of the null hypothesis of equal means

across the groups cannot be rejected at the 5% significance level for both the full and

preferred models. Furthermore, a Kruskall-Wallis test on the efficiencies from the

'inter-envelope' (see Charnes et al 1981) indicates that the efficiency distributions for

the three groups of universities do not differ significantly for the full model (χ2 = 1.07

with an associated p-value of 0.59). These results are confirmed for the reduced model

( χ2 = 2.55 with an associated p-value of 0.28). Thus, while the levels of inputs and

outputs clearly differ between types of HEI, the efficiency with which inputs are

transformed into outputs is not significantly different.

16

One aspect of efficiency which has not yet been examined is scale efficiency6.

The DEA is performed with constant returns to scale (CRS) and the results compared

to the VRS model. Scale efficiency is then calculated as the ratio of the CRS

efficiencies to the VRS efficiencies (see table 4 for full results). Scale efficiency is

high with an average of 96% (regardless of model). Again there is no significant

difference in mean scale efficiency between the three subgroups at the 5%

significance level.

An advantage of DEA compared to the parametric alternatives which has not yet

been highlighted is the wealth of managerial information provided by the technique.

The outcome of any DEA, in addition to efficiency scores, is a list of the 'peers' which

each inefficient DMU should ideally emulate in order to become efficient. This

information has a number of uses. It provides inefficient DMUs with institutions

whose practices it should try to emulate. In addition, the frequency with which an

efficient DMU appears as a peer is of interest: a low frequency suggests that it has an

extreme characteristic (for example size) which makes it an unsuitable peer to emulate

(Athanassopoulos & Shale 1997). Such universities may be deemed efficient because

of their 'extreme' characteristic. It is clear from table 5 that a number of efficient HEIs

have a low peer frequency. Around 20-25% of efficient HEIs do not appear in the

peer group of any inefficient HEI, and the efficiency scores of these DMUs, in

particular, should therefore be treated with a degree of caution.

<Table 5 here>

Given the high level of efficiency observed in English universities further

examination would be helpful to assess whether the differences in efficiency observed

across the individual HEIs are significant. Bootstrapping procedures (Simar & Wilson

6 It should be noted that the null hypothesis of CRS is rejected at the 5% significance level using both Banker's (1996) exponential and half normal tests.

17

1998; 1999; 2004b) are used to estimate 95% confidence intervals for each HEI's

efficiency score7, and these are illustrated for the reduced model in Figure 2 (the

results for the full model are very similar and therefore not reported). The plot of

efficiency scores8 and confidence intervals is notable for the fact that there is no

overlap between the 22 lowest performing HEIs and those HEIs which have the

maximum efficiency score. This observation of a significant difference in efficiency

between the lowest and highest performing HEIs has been observed elsewhere using

alternative output measures and techniques of analysis (Smith et al 2000; Smith &

Naylor 2001).

<Figure 2 here>

5. CONCLUSIONS

This paper has provided an overview of methods which might be used to assess

efficiency in higher education. DEA has the advantage over alternative (parametric)

methods that it can be applied in a multiple input multiple output production context.

The downside, however, is that, in its basic form, there are no significance tests for

comparing models, or for comparing the efficiency scores of individual or groups of

DMUs. Developments of the DEA approach which attempt to overcome these

drawbacks have been presented and illustrated using a data set of English universities.

Specific extensions which have been considered include the Pastor et al (2002) test

for assessing the relevance of input(s) and/or output(s) included in a DEA; the

Charnes et al (1981) method for testing for significant differences in the efficiency

distributions of different subgroups; and bootstrapping procedures (Simar & Wilson

1998; 1999) for deriving confidence intervals for the efficiency scores of individual 7 The confidence intervals are estimated according to the procedure described in Appendix 2 using a Fortran programme (Johnes 2004). A bandwidth of h = 0.02 is used to produce the results in Figure 2, but values of 0.01 and 0.05 give broadly similar results. 8 It should be noted that the efficiency scores have not been bias corrected, in line with Simar & Wilson's (2004b) caveat that the bias-correction be used with caution.

18

DMUs. The conclusions of the application of DEA to English universities are as

follows:

• The level of efficiency in English universities is high. This result is in

line with other studies of efficiency in tertiary education, but is

somewhat surprising given the lack of profit motivation typical of this

sector.

• The Pastor et al (2002) test is useful, to an extent, in reducing an input

output set to a smaller 'significant' set. Those wishing to compare nested

models should be aware that the Pastor et al test is concerned with

whether changes in the values of the efficiency scores (rather than

changes in the ranking of DMUs) are significant. Spearman's rank

correlation coefficient can be used in addition to the Pastor et al test to

provide complementary information.

• The Charnes et al (1981) procedure finds no significant differences in

the distribution of efficiencies for pre-1992, post-1992 and SCOP HEIs

in England. This is a surprising result given the obvious differences

between these groups in terms of their inputs and outputs, but suggests

that the efficiency with which inputs are converted into outputs does not

differ significantly across the subgroups, and that there are no efficiency

disadvantages in having diversity of provision in higher education. The

Charnes et al (1981) procedure could also be used to test for differences

between other possible definitions of subgroups: in USA higher

education, for example, it would be appropriate for testing for

differences between public versus private institutions; or between

19

subgroups based on a college's National Collegiate Athletic Association

(NCAA) division.

• While no differences emerge between HEI types in terms of efficiency

with which inputs are converted into outputs, the bootstrapping estimates

of the 95% confidence intervals for the efficiency scores of the reduced

model suggest that the difference in efficiency between the worst- and

best-performing English HEIs is significant. Thus, while DEA cannot

reliably be used to discriminate between the middle-performing HEIs in

terms of their level of efficiency, it can discriminate between the worst-

and best-performing HEIs.

20

APPENDIX 1: THE PASTOR ET AL (2002) TEST OF NESTED DEA

MODELS

The test works as follows. Denote the vector of s outputs used by DMU j (j = 1,

… ,n) by yj and the vector of m inputs used by DMU j by xj. The test of how relevant

is a variable is based on the ratio of the efficiency score for each DMU (j = 1, …, n)

for the full model, (denoted by ),(ˆ ''jj yxD , 1),(ˆ0 '' ≤≤ jj yxD ) to the efficiency score

for the reduced model (denoted by ),(ˆjj yxD , 1),(ˆ0 ≤≤ jj yxD ), where the reduced

model must be nested within the full model. Let ),(ˆ),(ˆ ''jjjjj yxDyxD=ρ (j = 1, …

,n) be observed values of a random sample jΓΓ ,...,1 drawn from a population

( )F,1~Γ (where F is a cumulative density function on [ )∞,1 ) and define

>Γ=

otherwise 0

if 1 ρjjT j = 1, …, n

where 1>ρ . The impact of the variable(s) excluded from the full model is considered

relevant if 0)( pP >>Γ ρ where 10 0 << p . In order to test the null hypothesis that

00 )(: pPH ≤>Γ ρ the value p is calculated as )1(1)( −−=>= oB

o TFTTPp ,

where ∑=

=n

jjTT

1

, To is the observed value of T, and, under H0,

),1(Binomial~ 0pnT − and BF is the cumulative density function of the

),1(Binomial 0pn − . Thus a small p-value suggests the null hypothesis should be

rejected. Pastor et al (2002) find that the test performs well for the values of 1.1=ρ

and 15.00 =p .

21

APPENDIX 2: BOOTSTRAPPING PROCEDURE

The bootstrapping procedure adopted here is derived from Simar and Wilson

(1998; 1999; 2004b). Denote the vector of s outputs used by DMU j (j = 1, … ,n) by yj

and the vector of m inputs used by DMU j by xj. The steps are as follows:

Step 1: Estimate the efficiency scores for the data set and reflect the data

DEA is applied to the given data on inputs and outputs to obtain an estimate of

efficiency for each DMU in the set, and this is denoted by ),(ˆjj yxD . These estimates

are reflected around unity by computing ),(ˆ2 jj yxD− for each ),(ˆjj yxD , j = 1, …

,n, providing 2n observations in total.

Step 2: Derive bootstrap values

a) Set a bandwidth h for use in the drawing of the bootstrap values (see

Simar & Wilson 1998 for further details on setting the bandwidth).

b) Draw n independently and identically distributed observations (denoted

by njj ,...,1 * =ε ) from the probability density function used as the

kernel distribution (the uniform distribution in this case).

c) Draw n values (denoted by njd j ,...,1 = ) independently and uniformly

from the set of 2n reflected distance function estimates. From these,

calculate the mean:

nddn

jj∑

=

=1

(A2.1)

and

( ) ( )dhdshdd jjj −+++=−

ε21

22* 1 (A2.2)

where 2s is the sample variance of jjj hdv ε+= .

d) Calculate the bootstrap values ( *ˆjD ) as

otherwise 2

1 if ˆ**

*

≤

= *j

jjj -d

ddD (A2.3)

Step 3: Define the pseudo data and obtain the bootstrap estimates of the efficiencies

Define a pseudo data set with input and output vectors (denoted by ),( **jj yx ) as

22

),(ˆ/**jjjjj yxDyDy = (A2.4)

jj xx =* (A2.5)

Obtain a value of B (B = 1000 is used in the analysis of section 4) bootstrap

estimates of the efficiency score for each DMU j (j = 1, … ,n) by applying DEA to the

pseudo data B times. These bootstrap estimates can be denoted for DMU k by

{ }Bbkkb yxD 1* ),(ˆ

= .

Step 4: Compute estimated confidence intervals for the efficiency scores

The 100 )1( α− % confidence interval for the true efficiency for DMU k, is

calculated by finding the values αα ab , such that:

ααα −=−≤−≤− 1)),(),(ˆPr( ayxDyxDb kkkk (A2.6)

The values αα ab , are not known but are estimated from the bootstrap estimates

{ }Bbkkb yxD 1* ),(ˆ

= by sorting the values ),(),(ˆ *kkkkb yxDyxD − in increasing order and

deleting )2/100( α % of the observations at each end of this list. Thus estimates of

αα ab −− and (denoted by αα ab ˆ and ˆ −− ) are the endpoints of the remaining array of

values such that αα ba ˆˆ ≤ . The bootstrap approximation of equation (A2.6) is therefore

ααα −≈−≤−≤− 1)ˆ),(),(ˆˆPr( ayxDyxDb kkkk (A2.7)

and so the estimated 100 )1( α− % confidence interval for the efficiency score of

DMU k is found by evaluating:

[ αα byxDayxD kkkkˆ)(ˆ,ˆ)(ˆ ++ ] (A2.8)

REFERENCES Abbott, M & Doucouliagos (2003) ‘The efficiency of Australian universities: a data

envelopment analysis’ Economics of Education Review, 22(1) pp89-97 Athanassopoulos, A D & Shale, E (1997) ‘Assessing the comparative efficiency of

higher education institutions in the UK by means of data envelopment analysis’ Education Economics, 5 (2) pp117-134

Avkiran, N K (2001) ‘Investigating technical and scale efficiencies of Australian universities through data envelopment analysis’ Socio-Economic Planning Sciences, 35(1) pp57-80

Banker, R D (1996) 'Hypothesis tests using data envelopment analysis' Journal of Productivity Analysis 7 pp139-159

Banker, R D, Charnes, A & Cooper, W W (1984) ‘Some models for estimating technical and scale inefficiencies in data envelopment analysis’ Management Science, 30(9) pp1078-1092

Byrnes, P & Valdmanis, V (1994) 'Analyzing technical and allocative efficiency of hospitals' in A. Charnes, W. Cooper, A. Lewin & L. Seiford (eds) Data Envelopment Analysis: Theory, Methodology and Applications, Kluwer, Boston

Charnes, A, Cooper, W W & Rhodes, E (1978) ‘Measuring the efficiency of DMUs’ European Journal of Operational Research, 2 pp429-444

Charnes, A, Cooper, W W & Rhodes, E (1981) 'Evaluating program and managerial efficiency: an application of DEA to program follow-through' Management Science, 27(6) pp668-697

Coelli, T & Perelman, S (1999) 'A comparison of parametric and non-parametric distance functions: with application to European railways' European Journal of Operational Research, 117 pp326-339

Coelli, T, Rao, D S & Battese, G E (1998) An Introduction to Efficiency and Productivity Analysis, Kluwer Academic, Norwell, Massachusetts

Cubbin, J & Tzanidakis, G (1998) 'Regression versus data envelopment analysis for efficiency measurement: an application to the England and Wales regulated water industry' Utilities Policy, 7 pp75-85

Dantzig, G B (1951) ‘Maximization of a linear function of variables subject to linear inequalities’ in T C Koopmans (ed) Activity Analysis of Production and Allocation, Wiley, New York

Färe, R, Grosskopf, S & Lovell, C A K (1994) Production Frontiers, Cambridge University Press, Cambridge

Farrell, M (1957) ‘The measurement of productive efficiency’ Journal of the Royal Statistical Society, Series A, 120 pp253-281

Førsund, F R, Lovell, C A K & Schmidt, P (1980) ‘A survey of frontier production functions and of their relationship to efficiency measurement’ Journal of Econometrics, 13 pp5-25

Førsund, F R & Hemaes, E (1994) 'A comparative analysis of ferry transport in Norway' in A. Charnes, W. Cooper, A. Lewin & L. Seiford (eds) Data Envelopment Analysis: Theory, Methodology and Applications, Kluwer, Boston

Grosskopf, S (1996) ‘Statistical inference and non-parametric efficiency: a selective survey’ Journal of Productivity Analysis, 7 pp161-176

Izadi, H, Johnes, G, Oskrochi, R & Crouchley, R (2002) ‘Stochastic frontier estimation of a CES cost function: the case of higher education in Britain’ Economics of Education Review, 21 pp63-71

24

Johnes, G (2004) CIDEA: software for evaluating confidence intervals in DEA problems, mimeo, Lancaster University Management School, available from http://www.lancs.ac.uk/people/ecagj/cidea1.html.

Johnes, J (2004) 'Efficiency measurement' in Johnes, G & Johnes, J (eds) The International Handbook on the Economics of Education, Edward Elgar, Cheltenham

Johnes, J (forthcoming) 'Measuring teaching efficiency in higher education: an application of data envelopment analysis to Economics graduates for UK universities' European Journal of Operational Research

Johnes, J & Taylor, J (1990) Performance Indicators in Higher Education: UK Universities, Open University Press and The Society for Research into Higher Education, Milton Keynes

Korhonen, P, Tainio, R & Wallenius, J (2001) ‘Value efficiency analysis of academic research’ European Journal of Operational Research, 130 pp121-132

Lovell, C A K (1993) 'Production frontiers and productive efficiency' in H O Fried, C A K Lovell & S S Schmidt (eds) The Measurement of Productive Efficiency, Oxford University Press, Oxford, pp3-67

Lovell, C A K & Pastor, J T (1997) 'Target setting: an application to a bank branch network' European Journal of Operational Research, 98 pp290-299

Mancebon, M J & Bandrés, E (1999) 'Efficiency evaluation in secondary schools: the key role of model specification and of ex post analysis of results' Education Economics, 7(2) pp131-152

Mancebon, M J & Mar Molinero, C (2000) ‘Performance in primary schools’ Journal of the Operational Research Society, 51(7) pp843-854

McCarty, T A & Yaisawarng, S (1993) ‘Technical efficiency in New Jersey school districts’ in H O Fried, C A K Lovell & S S Schmidt (eds) The Measurement of Productive Efficiency, Oxford, Open University Press, pp271-287

Paradi, J C, Vela, S & Yang, Z (2004) 'Assessing bank and bank branch performance: modelling considerations and approaches' in W.W. Cooper, L. M. Seiford & J. Zhu (eds) Handbook on Data Envelopment Analysis, Kluwer, Boston

Pastor, J T, Ruiz, J L & Sirvent, I (2002) 'A statistical test for nested radial DEA models' Operations Research, 50(4) pp728-735

Räty, T (2002) 'Efficient facet based efficiency index: a variable returns to scale specification' Journal of Productivity Analysis, 17 pp65-82

Ruggiero, J (1996) 'On the measurement of technical efficiency in the public sector’ European Journal of Operational Research, 90 pp553-565

Shephard, R W (1970) Theory of Cost and Production Functions, Princeton University Press, New Jersey

Simar, L & Wilson, P W (1998) 'Sensitivity analysis of efficiency scores: how to bootstrap in nonparametric frontier models' Management Science, 44 (1) pp49-61

Simar, L & Wilson, P W (1999) 'Performance of the bootstrap for DEA estimators and iterating the principle' Discussion Paper, Université Catholique de Louvain

Simar, L & Wilson, P W (2004a) 'Estimation and inference in two-stage, semi-parametric models of production processes' Discussion Paper, Institut de Statistique, Université Catholique de Louvain, Belgium

Simar, L & Wilson, P W (2004b) 'Performance of the bootstrap for DEA estimators and iterating the principle' in Cooper, W W, Seiford, L M and Zhu, J (eds) Handbook on Data Envelopment Analysis, Boston: Kluwer

25

Smith, J, McKnight, A & Naylor, R (2000) ‘Graduate employability: policy and performance in higher education in the UK’ Economic Journal, 110 F382-F411

Smith, J & Naylor, R (2001) ‘Dropping out of university: a statistical analysis of the probability of withdrawal for UK university students’ Journal of the Royal Statistical Society, Series A, 164(2) pp389-405

Figure 1: Diagrammatic representation of an output-oriented DEA

xy1

xy2

A B

E'

C

D

H

G

E

0

Figure 2: Efficiency score and 95% confidence intervals for English universities (reduced model)

HEI

Effic

iency

scor

e an

d 95

% co

nfide

nce

boun

ds1.4

1.2

1.0

.8

.6

.4

.2

0.0

Efficiency score

Lower bound

Upper bound

Table 1: Definition of input and output variables for the DEA

Variables Definition1

OUTPUTS: GRADQUAL2 Total number of first degrees awarded weighted by degree classification i.e.

GRADQUAL = (number of firsts * 30) + (number of upper seconds * 25) + (number of lower seconds * 20) + (number of thirds * 15) + (number of unclassifieds * 10)

POSTGRAD2 Total number of higher degrees awarded (includes both doctorate and other higher degrees).

RESEARCH3 Value of the recurrent grant for research awarded by the Higher Education Funding Council for England (HEFCE) in £.

INPUTS: UGQUAL2,5 Total number of FTE undergraduate students studying for a first degree

multiplied by the average A level points for first year full-time undergraduate students (A level score is averaged over 1994/95, 1995/96, 1996/97 and 1997/98. Note that A = 10, B = 8, C = 6, D = 4, E = 2).

PG2 Total number of FTE postgraduate students.

STAFF4 Total number of full-time academic staff for teaching or teaching and research or research only purposes.

CAPITAL4 Total depreciation and interest payable in £.

LIBCOMP4 Total expenditure on central libraries and information services, and on central computer and computer networks excluding academic staff costs and depreciation in £.

ADMIN4 Expenditure on central administration and central services excluding academic staff costs and depreciation in £.

Notes: 1. All data refer to the year 2000/01 with the exception of ASCORE. 2. Source: Students in Higher Education Institutions 2000/01, Higher Education Statistics Agency 3. Source: HEFCE www.hefce.ac.uk 4. Source: Resources of Higher Education Institutions 2000/01, Higher Education Statistics Agency 5. Higher Education Statistics Agency

Table 2: Descriptive statistics for the output and input variables

All universities

N = 109 Pre-1992 universities

N = 47 Post-1992 universities

N = 34 SCOP HEIs

N = 28

Variable mean standard deviation

mean standard deviation

mean standard deviation

mean standard deviation

GRADQUAL 43976.84 26947.79 44909.04 26482.24 63244.85 19791.30 19015.18 10769.85 POSTGRAD 628.17 617.99 1075.64 654.34 469.71 271.27 69.46 65.88 RESEARCH 7515477.27 13096930.28 16208330.36 16295283.26 1466685.82 1103071.16 268863.46 404732.31 UGQUAL 114016.87 84133.98 152671.60 99347.57 122996.04 44365.28 38228.86 20830.27 PG 1824.35 1307.57 2502.68 1407.37 1915.65 797.09 574.86 507.37 STAFF 823.35 811.00 1253.83 1029.07 750.00 232.00 189.82 123.81 CAPITAL 5962.71 5002.20 7847.57 6116.33 6854.97 2689.38 1715.36 1272.08 LIBCOMP 5311.28 4386.04 7192.70 5426.36 5825.79 1843.55 1528.39 1137.47 ADMIN 10793.53 7358.53 13939.51 8737.43 11813.77 3712.86 4273.93 2656.95

See text footnote 2 for information on pre-1992, post-1992 and SCOP HEIs.

30

Table 3: Comparing alternative specifications of the DEA model

Input & Output variables M0 M1 M2 M3 M4 M5 M6 M7 M8

GRADQUAL X X X X X X X X POSTGRAD X X X X X X X X RESEARCH X X X X X X X X

UGQUAL X X X X X X X X PG X X X X X X X X CAPITAL X X X X X X X X ADMIN X X X X X X X X LIBCOMP X STAFF X Spearman's r 0.92 0.46 0.75 0.69 0.78 0.81 0.72 0.77 Pastor et al p-value 0.99 0.00 0.05 0.00 0.00 0.03 0.00 0.00

31

Table 4: Efficiency scores for full and preferred models

University Name ID Full model Preferred model Full model Preferred model Technical efficiency Technical efficiency Scale efficiency Scale efficiency Overall Mean = 94.61 Overall Mean = 92.51 Overall Mean = 96.45 Overall Mean = 96.13 Pre-1992 HEIs Mean = 96.34 Mean = 94.25 Mean = 95.69 Mean = 95.07 Aston University 2 87.38 80.69 99.73 99.05 The University of Bath 4 83.12 70.20 97.88 99.81 The University of Birmingham 6 100.00 100.00 100.00 100.00 The University of Bradford 10 100.00 100.00 100.00 100.00 The University of Bristol 13 89.27 88.47 84.62 85.38 Brunel University 14 88.31 77.52 97.81 98.17 The University of Cambridge 16 100.00 100.00 90.83 90.83 City University 24 100.00 100.00 100.00 100.00 Cranfield University 26 100.00 100.00 100.00 100.00 University of Durham 31 100.00 97.94 95.02 93.90 The University of East Anglia 32 80.35 77.72 99.96 99.88 The University of Essex 35 99.96 99.94 99.39 99.39 The University of Exeter 36 93.34 86.57 98.31 95.62 Goldsmiths College 39 100.00 93.95 100.00 99.11 The University of Hull 45 100.00 100.00 100.00 100.00 Imperial College of Science, Technology & Medicine 46 100.00 100.00 100.00 100.00 The University of Keele 49 97.29 85.95 99.77 99.90 The University of Kent at Canterbury 50 88.84 83.01 99.94 99.87 King's College London 53 100.00 100.00 100.00 100.00 The University of Lancaster 55 100.00 100.00 100.00 100.00 The University of Leeds 57 100.00 100.00 91.70 91.70 The University of Leicester 58 100.00 98.71 90.11 91.29 The University of Liverpool 62 100.00 97.05 83.97 86.52 University of London (Institutes and activities) 64 100.00 100.00 100.00 100.00 London School of Economics and Political Science 67 100.00 100.00 100.00 100.00

32

Loughborough University 69 100.00 100.00 100.00 100.00 University of Manchester 71 98.53 98.53 82.26 82.26 Institute of Science & Technology The University of Manchester 72 84.17 82.69 99.92 97.41 The University of Newcastle-upon-Tyne 75 100.00 100.00 92.03 92.03 The University of Nottingham 83 100.00 100.00 84.54 84.54 The University of Oxford 86 100.00 100.00 88.17 88.17 Queen Mary and Westfield College 89 100.00 97.73 87.89 89.73 The University of Reading 91 100.00 100.00 100.00 100.00 Royal College of Music 96 100.00 100.00 100.00 69.79 Royal Holloway and Bedford New College 97 86.30 84.24 96.74 96.34 The Royal Veterinary College 99 100.00 100.00 100.00 100.00 St George's Hospital Medical School 100 74.60 74.60 91.53 91.53 The University of Salford 104 97.18 93.75 93.11 96.11 The School of Oriental and African Studies 105 100.00 100.00 100.00 100.00 The School of Pharmacy 106 100.00 100.00 100.00 100.00 The University of Sheffield 108 91.83 91.83 86.75 86.38 The University of Southampton 111 98.81 97.13 85.67 86.34 The University of Surrey 115 98.61 83.37 99.94 99.62 The University of Sussex 116 97.69 97.02 95.65 95.96 University College London 121 100.00 100.00 100.00 100.00 The University of Warwick 122 92.40 90.91 84.97 82.61 The University of York 130 100.00 100.00 99.08 99.02 Post-1992 Universities Mean = 94.25 Mean = 92.80 Mean = 95.96 Mean = 95.07 Anglia Polytechnic University 1 100.00 100.00 100.00 100.00 Bournemouth University 9 100.00 100.00 100.00 100.00 The University of Brighton 12 86.62 86.62 99.19 99.19 The University of Central England in Birmingham 19 91.94 91.94 99.99 99.99 The University of Central Lancashire 20 90.83 89.63 81.24 82.33

33

Coventry University 25 100.00 100.00 100.00 100.00 De Montfort University 29 91.74 86.90 81.62 84.40 University of Derby 30 95.64 95.64 95.73 95.73 The University of East London 33 88.96 88.96 99.90 99.90 The University of Greenwich 40 90.71 90.02 90.68 91.38 University of Hertfordshire 42 100.00 100.00 96.07 96.07 The University of Huddersfield 44 77.54 72.19 95.16 98.46 Kingston University 54 78.24 70.64 88.91 95.71 Leeds Metropolitan University 56 100.00 100.00 96.50 96.50 The University of Lincoln 59 100.00 100.00 100.00 100.00 Liverpool John Moores University 61 100.00 100.00 88.65 86.85 London Guildhall University 65 86.13 86.13 99.67 99.65 University of Luton 70 82.53 82.53 98.90 98.90 The Manchester Metropolitan University 73 100.00 100.00 100.00 100.00 Middlesex University 74 100.00 94.80 100.00 91.29 The University of Northumbria at Newcastle 80 100.00 100.00 98.48 98.48 The Nottingham Trent University 82 96.83 96.83 95.10 95.10 Oxford Brookes University 85 96.56 83.86 91.96 96.70 The University of Plymouth 87 100.00 100.00 96.57 96.47 The University of Portsmouth 88 100.00 100.00 100.00 100.00 Sheffield Hallam University 107 100.00 99.33 91.29 89.36 South Bank University 109 100.00 91.47 95.95 92.41 Staffordshire University 112 100.00 100.00 97.91 97.61 The University of Sunderland 113 92.55 89.98 93.13 91.38 The University of Teesside 117 85.80 85.80 98.61 98.61 Thames Valley University 118 71.80 71.80 99.44 99.44 University of the West of England, Bristol 123 100.00 100.00 100.00 100.00 The University of Westminster 124 100.00 100.00 100.00 100.00 The University of Wolverhampton 126 100.00 100.00 92.08 92.08

34

SCOP and SCOP-type HEIs Mean = 92.14 Mean = 89.27 Mean = 98.32 Mean = 98.06 Bath Spa University College 3 94.56 90.07 98.90 97.77 Bolton Institute of Higher Education 8 78.63 78.63 98.91 98.91 Bretton Hall College of HE 11 100.00 91.09 100.00 99.99 Buckinghamshire Chilterns University College 15 100.00 100.00 100.00 100.00 Canterbury Christ Church University College 18 63.54 63.54 99.78 99.78 Central School of Speech and Drama 21 100.00 92.34 92.67 71.79 Chester College of HE 22 100.00 100.00 100.00 100.00 College of St Mark and St John 101 100.00 100.00 100.00 100.00 Dartington College of Arts 28 100.00 100.00 87.53 85.42 Edge Hill College of Higher Education 34 85.36 85.24 99.68 99.82 Falmouth College of Arts 37 100.00 100.00 100.00 100.00 Harper Adams University College 41 87.29 87.29 99.21 99.21 Kent Institute of Art & Design 51 100.00 100.00 100.00 100.00 King Alfred's College, Winchester 52 90.50 82.88 99.64 99.72 Liverpool Hope 60 71.90 60.28 93.57 99.82 Norwich School of Art and Design 81 100.00 100.00 100.00 100.00 Southampton Institute 110 100.00 100.00 100.00 100.00 St Martin's College 102 74.90 72.97 99.49 99.79 St Mary's College 103 100.00 100.00 100.00 100.00 The London Institute 66 100.00 100.00 100.00 100.00 The Surrey Institute of Art and Design 114 100.00 100.00 100.00 100.00 Trinity And All Saints College 119 100.00 93.62 100.00 98.76 University College Chichester 23 91.54 90.14 99.49 99.62 University College Northampton 78 82.59 81.19 99.27 96.21 University College Worcester 127 100.00 100.00 100.00 100.00 University of Gloucestershire 38 75.35 74.45 99.77 99.93 University of Surrey, Roehampton 92 92.19 87.24 99.24 99.98 York St John College 129 91.54 68.48 85.74 99.26

35

Table 5: Frequency with which an efficient DMU appears as a peer (full and

final models)

University Name ID No. of times No. of times DMU is a peer DMU is a peer

Full model Preferred

model Pre-1992 Universities The University of Birmingham 6 2 4 The University of Bradford 10 2 4 The University of Cambridge 16 0 0 City University 24 1 2 Cranfield University 26 4 6 University of Durham* 31 0 Goldsmiths College* 39 1 The University of Hull 45 11 9 Imperial College of Science, Technology & Medicine 46 21 10 King's College London 53 0 0 The University of Lancaster 55 2 2 The University of Leeds 57 5 6 The University of Leicester* 58 0 The University of Liverpool* 62 1 University of London (Institutes and activities) 64 2 3 London School of Economics and Political Science 67 8 10 Loughborough University 69 0 2 The University of Newcastle-upon-Tyne 75 0 3 The University of Nottingham 83 0 0 The University of Oxford 86 4 12 Queen Mary and Westfield College* 89 0 The University of Reading 91 18 24 Royal College of Music 96 0 0 The Royal Veterinary College 99 0 0 The School of Oriental and African Studies 105 3 8 The School of Pharmacy 106 6 17 University College London 121 5 11 The University of York 130 0 0 Post-1992 Universities Anglia Polytechnic University 1 3 6 Bournemouth University 9 0 0 Coventry University 25 6 8 University of Hertfordshire 42 0 3 Leeds Metropolitan University 56 1 1 The University of Lincoln 59 34 25 Liverpool John Moores University 61 4 4 The Manchester Metropolitan University 73 10 8 Middlesex University* 74 4 The University of Northumbria at Newcastle 80 2 12 The University of Plymouth 87 0 0 The University of Portsmouth 88 17 15

36

Sheffield Hallam University* 107 1 South Bank University* 109 0 Staffordshire University 112 2 2 University of the West of England, Bristol 123 10 9 The University of Westminster 124 5 3 The University of Wolverhampton 126 0 0 SCOP and SCOP-type HEIs Bretton Hall College of HE* 11 7 Buckinghamshire Chilterns University College 15 17 12 Central School of Speech and Drama* 21 0 Chester College of HE 22 4 7 Dartington College of Arts 28 2 5 Falmouth College of Arts 37 7 7 Kent Institute of Art & Design 51 1 0 The London Institute 66 9 15 Norwich School of Art and Design 81 4 3 College of St Mark and St John 101 6 7 St Mary's College 103 2 3 Southampton Institute 110 3 5 The Surrey Institute of Art and Design, University College 114 9 8 Trinity And All Saints College* 119 2 University College Worcester 127 2 7 * HEI does not achieve an efficiency score of 1 in the final DEA model