prior-ratio-analysis procedure to improve data envelopment analysis for performance measurement

Prior-Ratio-Analysis Procedure to Improve Data Envelopment Analysis for PerformanceMeasurementAuthor(s): M. I. Gonzalez-BravoSource: The Journal of the Operational Research Society, Vol. 58, No. 9 (Sep., 2007), pp. 1214-1222Published by: Palgrave Macmillan Journals on behalf of the Operational Research SocietyStable URL: http://www.jstor.org/stable/4622803 .

Accessed: 28/06/2014 13:19

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp

.JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range ofcontent in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new formsof scholarship. For more information about JSTOR, please contact [email protected].

.

Palgrave Macmillan Journals and Operational Research Society are collaborating with JSTOR to digitize,preserve and extend access to The Journal of the Operational Research Society.

http://www.jstor.org

This content downloaded from 141.101.201.32 on Sat, 28 Jun 2014 13:19:11 PMAll use subject to JSTOR Terms and Conditions

http://www.jstor.org/action/showPublisher?publisherCode=pal

http://www.jstor.org/action/showPublisher?publisherCode=ors

http://www.jstor.org/stable/4622803?origin=JSTOR-pdf

http://www.jstor.org/page/info/about/policies/terms.jsp


Journal of the Operational Research Society (2007) 58, 1214-1222 O 2007 Operational Research Society Ltd. All rights reserved. 0160-5682/07 $30.00

www.palgravejournals.com/jors

Prior-Ratio-Analysis procedure to improve data

envelopment analysis for performance measurement

MI Gonzalez-Bravo* Universidad de Salamanca, Salamanca, Spain

This paper proposes the use of a Prior-Ratio-Analysis procedure, analysing output/input ratio indicators, allowing the improvement in efficiency measurement by means of data envelopment analysis (DEA) methodology. This prior analysis is based on the existence of a relationship of individual ratio in the firms to DEA efficiency scores. Use of the proposed procedure allows (i) detection of efficient units whose efficiency could be overestimated and (ii) identification of certain inputs/outputs enhancing particular behaviours. Accordingly, the DEA efficiency analysis could be improved with a major understanding about the factors determining the unit efficiency, and with a measure as a true indicator for discriminating between units, and for ranking them. Journal of the Operational Research Society (2007) 58, 1214-1222. doi:10.1057/palgrave.jors.2602247 Published online 11 October 2006

Keywords: data envelopment analysis; performance measurement; ratio analysis

1. Introduction

Data envelopment analysis (DEA) is a method to asses the effi-

ciency of analysed units, by managing multiple inputs/outputs, that generates a measure of efficiency that reflects 'overall' firm's performance. However, this measure should not be considered either absolute or accurate, since DEA results are sensitive to particular prior issues. DEA efficiency scores are dependent on at least both the sample size and the number of variables. In fact, it has been described that DEA results could be biased, efficiency could be overestimated, units could be erroneously classified as efficient or inefficient, and a proper ranking or classification cannot be obtained (Smith, 1997; Bowlin, 1998; Cubbin and Tzanidakis, 1998a; Zhang and Bartels, 1998; Sarkis, 2000; Staat, 2001; Jenkins and Anderson, 2003).

A growing number of works apply DEA as a tool to rend accurate results about the efficiency of different decision mak-

ing units (DMUs), that is, sectors, countries or firms. After

having chosen a performance approach and the appropriate input/output variables, it is common to apply the so-called global DEA-model (GDM) that includes all these selected variables. The efficiency measure obtained is usually regarded as a true indicator for discriminating between units and for

ranking them. However, the results of a single model for evaluating the

firms' efficiency are not necessarily a good indicator for set- ting priorities among behaviours (Talluri and Yoon, 2000),

neither for the efficient nor for the inefficient companies. All the units identified as efficient are scored unity in the DEA and this fact prevents further discrimination among them. Fur- thermore, a firm could be erroneously labelled as efficient (Smith, 1997) and therefore cannot be a proper reference for the rest (Cubbin and Tzanidakis, 1998b; Talluri, 2000a). On the other hand, regarding the inefficient units, which score below unity, it is only possible to conclude that they perform poorly, since they fail to achieve maximum efficiency despite the freedom for choosing the optimal weights (Sinuany-Stern and Friedman, 1998).

We prove in this paper that the use of the GDM results in two kinds of errors with regard to the efficiency measure. On the one hand, it is possible to overestimate the efficiency of a firm that performs high in a single dimension. On the other hand, it is possible to underestimate the efficiency of a firm that performs reasonably well in all the considered dimensions but that does not stand out in any of them. These are the main limitations that prevent using the GDM to explain the performance and efficiency of the firms and to correctly discriminate among them.

This paper surveys and proposes an alternative way, based on a Prior-Ratio-Analysis (PRA), that allows the identification of atypical behaviours while providing insights into the factors that determine the unit efficiency. This PRA is based on the relationships between the individual ratios of the firms and the corresponding DEA efficiency scores. The identification of such relationships through a PRA allows (i) to an- ticipate efficient units that conform the production possibility set and (ii) to detect efficient units whose efficiency could be overestimated because of an unusual variable weighting.

* Correspondence: MI Gonzalez-Bravo, Facultad Economia y Empresa,

Campus Miguel de Unamuno. Edificio FES, Salamanca 37007, Spain. E-mail: [email protected]



MI Gonzalez-Bravo-Prior-Ratio-Analysis procedure 1215

Previous reports have described the relationship DEA/ output-input ratio analysis displaying the differences and the similarities of both approaches to assess efficiency and to rank units (Smith, 1990; Fernandez-Castro and Smith, 1994; Thanassoulis et al, 1996; Bauer et al, 1998; Chen and Yeh, 1998; Worthington, 1998; Avkiran, 1999; Zhu, 2000). Most of these works conclude that both methodologies are comple- mentary and should be applied simultaneously. The PRA here proposed provides an easy way to integrate both approaches.

The PRA starts with the initial ranking of the firms according to some individual ratios. This initial classification allows detecting likely efficient units and recognizing their weaknesses and strengths while identifying certain inputs and/or outputs that enhance specific behaviours. With this information, it is possible to construct simplified DEA models that combine different input/outputs sets as variables in order to focus onto the detected-specific behaviours. The parallel analysis of various of these simplified DEA models points out not merely overall similarities/dissimilarities, but also highlights particular aspects that would otherwise have gone unnoticed under other approaches.

The selection of different input/outputs combinations as DEA alternative models to obtain different measures of efficiency is not a new idea. The elimination of an input and/or output for analysing the effect on the measured efficiency has been already considered in several papers. The simple correlation analysis between variables and efficiency scores has served as a criterion to identify significant variables but there is no evidence to assert that there is always a positive relation between high correlated variables and changes in efficiency (Norman and Stoker, 1991; Dyson et al, 2001; Jenkins and Anderson, 2003). The PRA method proposed here is easily ap- plicable for selecting sensible combinations of inputs/outputs and for generating accurate results both in ranking the firms and in identifying specific behaviours. The selected DEA alternative models result in a measure able to represent the efficiency of the firms as a multi-component variable, which correlates with different efficiency indicators and is suitable for generalization.

2. The Weaknesses of a GDM to evaluate the efficiency

The score obtained after a GDM could fail to capture particular efficiency factors and to identify atypical behaviours. This section analyses the two weaknesses of a GDM, namely the overestimation of partial efficiencies and the underestimation of global efficiencies.

As for the overestimation of partial efficiencies, certain units could score highly due to particular combinations of the considered variables. This is a consequence of DEA flexibil- ity in assigning the most favourable weights to both inputs and outputs in order to maximize the unit's performance (Talluri, 2000b; Talluri and Yoon, 2000; Dyson et al, 2001; Jenkins and Anderson, 2003; Allen and Thanassoulis, 2004). Accordingly, a particular input/output combination might be

positively overvalued, overweighting certain variables and rendering an efficiency overestimation (Talluri, 2000b). In these conditions, a given company may be identified as efficient as long as it is not dominated because no other firms or linear combination of firms score higher (Bauer et al, 1998; Cooper et al, 2001). As for the underestimation of global efficiencies, a firm that shows equilibrated behaviour in every dimension can be overlooked favouring extreme atypical behaviours or partial efficiencies.

In order to make them evident the weaknesses of this GDM, we have applied a DEA CCR-output-oriented model to the efficiency analysis of 31 companies. These firms have been selected among the largest thousand Spanish firms and oper- ate in the same sector of activity NACE 5 digit. This prior requirement for the selection, of belonging to a very specific code of activity, guarantees both a high homogeneity between the selected DMUs and the absence of external environmental influences. The specification of inputs and outputs is made according to a profitability-productivity approach. Thus, an efficient firm in this instance tends to minimize the economic and productive resources associated, while maximizing results as profit, revenues and value-added. According to this approach, we have selected three inputs and three outputs as variables usually linked with the performance measures considered in the analysis of both profitability and productivity of a firm. The input variables are fixed assets, number of employees and total assets, and the output variable are profit (before tax and before interest expenses) value-added and sales revenue.

The three inputs and the three outputs generate nine possible single output-to-input combinations as individual ratios in a traditional one-dimensional analysis. These measures allow to evaluate the units in nine dimensions of performance and to rank them in nine different ways, but without a prior possibility to create a multiple global measure (Greenberg and Nunamaker, 1987; Fernandez-Castro and Smith, 1994; Thanassoulis et al, 1996; Chen and Yeh, 1998). In order to provide a global performance measure, we have defined an additional variable, named RANK_9. This variable ranks the firms according to the average position of each company along the nine ranks. The individual indicators for each of the firms are listed in Table 1.

The selected variables also generate 49 input/output combinations that could be analysed using a DEA approach resulting in 49 efficiency scores. This provides us with a wide range of efficiency measures to assess the firm's performance. The GDM is obviously one of these feasible combinations, which results in a measure of efficiency that includes all the selected input and output variables. The 48 additional efficiency scores will highlight punctual discrepancies between global and particular behaviours, depending on the number and/or combinations of both inputs and outputs (Serrano-Cinca and Mar Molinero, 2004).

The GDM analysis resulted in six companies scored unity and labelled as efficient. These results provide, however, no information about the possible differences among the six



1216 Journal of the Operational Research Society Vol. 58, No. 9

Table 1 Rank positions for each firm in the nine individual indicators selected

Firm 1/A 1/B 1/C 2/A 2/B 2/C 3/A 3/B 3/C RANK_9

DMU1 24 24 24 18 16 12 19 19 19 20 DMU2 12 13 12 15 13 24 5 4 3 9 DMU3 10 12 7 16 19 7 17 24 11 14 DMU4 29 30 29 22 22 17 20 15 17 26 DMU5 1 2 2 8 1 10 7 3 8 1 DMU6 18 23 19 11 25 4 16 31 15 16 DMU7 30 31 31 21 28 16 22 30 21 31 DMU8 15 5 10 24 2 19 21 7 20 13 DMU9 6 4 3 9 9 3 15 21 18 7 DMU10 22 16 15 27 12 18 30 23 27 25 DMU11 3 3 1 14 3 8 14 9 12 4 DMU12 19 15 11 28 20 20 26 18 22 23 DMU13 11 14 14 12 17 29 4 5 7 10 DMU14 28 28 28 17 21 15 12 12 13 19 DMU15 25 18 21 26 6 11 31 25 24 24 DMU 16 14 11 13 20 5 26 23 14 30 15 DMU17 5 7 5 3 4 1 8 13 5 2 DMU 18 23 1 18 31 15 31 28 I 28 21 DMU19 26 26 26 29 11 28 29 17 31 30 DMU20 8 22 25 1 30 30 1 2 1 11 DMU21 4 10 9 5 18 22 3 6 4 5 DMU22 20 19 22 23 29 25 24 29 26 29 DMU23 27 27 27 30 10 27 27 10 25 28 DMU24 16 25 23 4 23 14 10 27 23 17 DMU25 31 29 30 6 24 5 9 26 10 18 DMU26 17 17 16 25 27 23 25 28 29 27 DMU27 13 20 17 7 31 21 2 8 2 12 DMU28 21 21 20 19 26 13 18 22 16 22 DMU29 2 9 4 2 8 2 6 16 6 3 DMU30 7 8 6 10 14 6 13 20 14 8 DMU31 9 6 8 13 7 9 11 11 9 6

Shadowed cells highlight the firms scoring best for the corresponding indicator. The individual indicators are named according to the ratio output/input that represent, as follows: 1/A (profit/fixed assets), 1/B (profit/number of employees), 1/C (profit/total assets), 2/A (value-added/fixed assets), 2/B (value-added/number of employees), 2/C (value-added/total assets), 3/A (sales revenue/fixed assets), 3/B (sales revenue/number of employees) and 3/C (sales revenue/total assets).

Rank_9 defines the position of the given company in the ranking that gathers the results of the nine individual indicators.

efficient units, or about particular high- or low-efficiency values regarding concrete inputs/outputs variables. In fact, two DMUs may obtain the same DEA score despite the fact that they can differ in the achievement of efficiency (Serrano- Cinca et al, 2005).

There were significant differences in the behaviour of the six efficient firms when considering additional criteria, as shown in Table 2 . These results point out again the two main weaknesses of the GDM discussed above: the limitation in

identifying both partial efficiencies and global efficiencies. The former is the case of unit DMU18, classified as efficient in 50% of the models and with an average efficiency value of 0.58, therefore oscillating between being efficient or highly inefficient. The latter is the case of unit DMU29 with an average efficiency value of 0.78. This unit was seldom classified as efficient (24%) but displayed an equilibrated behaviour (third position in RANKS9) and was near to the frontier in most of the models.

The results of DMU18 and DMU29 show that there is no evident link between the RANK_9 variable and the number

of times a firm is classified as efficient. Moreover, according only to the individual ratios, units DMU18 and DMU20 would not be evaluated as efficient, since they score even below other units that the global DEA classifies as inefficient. On the contrary, there is a positive correlation between the number of indicators for which a given firm scores best (Top-position) and the number of times the same firm is classified as efficient (Times efficient) (r = 0.929, P < 0.01). This relation will be a key issue for the development of the procedure proposed in this paper.

These results prove that some firms display atypical behaviours and that misevaluations of the efficiency could arise when considering only a GDM analysis. The main assump- tion is that a proper selection of the models to be tested can provide reliable information without losing deepness in the in- terpretation. Different input/outputs combinations should allow justifying the choice efficient/inefficient, while preventing particular or anomalous behaviour, even if it means skipping certain variables. A reasonable alternative could be the elimination of some inputs and/or outputs. However, if a single



MI Gonzalez-Bravo---Prior-Ratio-Analysis procedure 1217

Table 2 Global description of the results achieved throughout the different methods analysed

Times considered efficient Efficiency average TOP-Position Efficient Firms (49 models) % (49 models) individual indicators RANKS

DMU5 30 61.22 0.87 2 1 DMU 11 16 32.65 0.74 1 4 DMU17 16 32.65 0.74 1 2 DMU18 24 48.97 0.58 2 21 DMU20 32 65.30 0.75 3 11 DMU29 14 23.57 0.78 0 3

The six efficient DMUs considered are the firms labelled as efficient by the GDM including the three inputs and the three outputs. Times efficient gathers the number of times a firm has been labelled efficient using the DEA approach out of the 49 feasible solutions. Efficiency average is the average value of the efficiency scores that a firm has obtained in the 49 DEA solutions. This measure could be regarded as an averaged multidimensional analysis. Top-Position is the number of individual indicators in which the firm scores best. Rank9 is the rank position of each firm in the later column of Table 1. This measure could be used like an averaged multiple one-dimensional analysis.

input and/or output is arbitrarily eliminated by the analysis, the probability of misidentifying a unit as efficient does not necessarily decrease. As an example, DMU18 could be considered as a poorly efficient performer (0.58 efficiency average) but will be considered efficient in 24 of 49 combinations. These results point out that the inputs/outputs combinations must be carefully selected and with a sensible procedure.

3. A PRA procedure for identifying significant input/output variables and selecting DEA alternative models

Table 3 shows the firms labelled efficient according to different input/output combinations. The special features in the performance of efficient firms can be revealed with the analysis of the results in the 49 DEA models. The question arises then, whether an a priori identification of the particular behaviours is possible without resorting to resolve all the input/output combinations in a generalized DEA analysis.

In a data set of j DMUs, j= 1, 2, ..., n, s outputs, yrj, r= , 2, ..., s and m inputs, xij, i= 1, 2, ..., m, the CCR model

(Charnes et al, 1978), can be expressed as

Sr=1 UrYro

max l lViXio

s.t. Iurr i j 1, j = 1 2 ... , , Vi Ur 0

In this model, Chen and Ali (2002) showed that a DMUk, where Yrk/Xik = max j[yrj/xij] for any i = 1 ... m and r = 1 ... s, is located on the constant returns to scale frontier, thus describing the relationship between ratio analysis and DEA efficiency.

The simplest DEA model-a single input and a single output-renders a sole unit labelled as efficient, which is pre- cisely that with a higher ratio for the individual indicator con- structed with the input/output related in this simplest DEA model (Dyson et al, 2001). Moreover, the solutions reached for each and all of these simple models belong also to the possible solutions of a DEA model including all inputs/outputs

variables (Chen and Ali, 2002). Interestingly, this fact is also true for any model that includes simultaneously both the considered input and output as variables.

We define a DMUk with a highest ratio for the individual indicator Yr'k/Xi'k. The DMUk will be efficient for any DEA model with m inputs, i = 1, ..., m, and s outputs, r= 1,

..... s,

when both, i' and r' are included simultaneously on the m x s input/output variables.

It is known in this sense that the optimal DEA solution for the efficient unit is not unique. Thus, for a selection of weights corresponding to the m x s model is given by

S1 for i = i' and 1 forr=r' ( V =,and

ur= (1)

i=-0 for i#:ir 0 forr:r'

Let I C {1 ..., m} be a subset of inputs and R C {1 ..., s} a subset of outputs. If i' e I and r' e R, then DMUk will be efficient for any solution of weights satisfying (1). Thus, DMUk will be efficient for any I and any R that include simultaneously the i' input and the r' output.

This means that a unit ranked in the best position in an individual output/input ratio will also be efficient for every DEA model that includes simultaneously these input and output. A prior analysis of the rankings in the individual ratios will allow us identifying (i) the models in which a firm will be efficient; (ii) the models with a higher number of DMUs labelled efficient and (iii) the critical input(s)/output(s) for each firm. However, it should be considered that a firm that is not scored best for any individual ratio could be labelled as efficient by a model if (Chen and Ali, 2002)

msr = max j r=1 Yr(2) EM=l Xik iC

l X1ij

Thus, the firms scoring in the second place for a given ratio could satisfy (2) and could be classified as efficient in the models that include such ratio. DMU29 provides a good example in this sense, since this firm does not score best in any individual indicator but it is classified as efficient in 14 of 49 models.




Table 3 Efficiency values after the 49 input/output combinations considered

Model Efficient units Average efficiency Efficient DMUs Model Efficient units Average efficiency Efficient DMUs

ABC 123 6 0.742 5,11,17,18,20,29 ABC3 2 0.445 18,2 ABC23 5 0.734 5,17,18,20,29 BC3 2 0.442 18,2 BC 123 5 0.733 5,11,17,18,20 B123 2 0.426 5,18 BC23 4 0.725 5,17,18,20 B23 2 0.425 5,18 ABCI2 6 0.706 5,11,17,18,20,29 A123 3 0.421 5,20,29 ABC2 4 0.678 5,17,20,29 A12 3 0.419 5,20,29 AC123 5 0.664 5,11,17,20,29 B12 2 0.411 5,18 BC12 4 0.656 5,11,17,18 B2 1 0.390 5 C123 3 0.654 11,17,20 AB3 2 0.378 18,2 AC23 3 0.633 17,20,29 C3 1 0.366 20 AB 123 4 0.628 5,18,20,29 AC3 1 0.366 20 BC2 2 0.627 5,17 ABCI 3 0.359 5,11,18 C23 2 0.626 17,2 A23 1 0.341 20 AB23 4 0.623 5,18,20,29 A2 1 0.341 20 AC12 5 0.613 5,11,17,20,29 A13 2 0.329 5,2 AB12 4 0.612 5,18,20,29 AC1 2 0.329 5,11 AB2 3 0.587 5,20,29 AB1 2 0.326 5,18 AC2 3 0.578 17,20,29 BC 3 0.304 5,11,18 ABC13 4 0.575 5,11,18,20 Al 1 0.290 5 BC13 4 0.568 5,11,18,20 Cl 1 0.266 11 C12 2 0.556 11,17 B13 1 0.184 18 AC13 3 0.528 5,11,20 BI 1 0.147 18 C2 1 0.526 17 B3 1 0.136 18 C13 2 0.521 11,2 A3 1 0.092 20 AB13 3 0.475 5,18,20

Shaded cells encompass the 49 models tested. The feasible models have been named according to the inputs and outputs included, as follows: A (fixed asset), B (number of employees), C (total asset) for the inputs; and 1 (profit), 2 (value-added), 3 (sales revenue), for the outputs. The models are listed using the average efficiency score in decreasing order. The average efficiency is calculated as the average of the efficiency scores that each firm obtains in the model. The number of firms labelled efficient by a model is showed in the column Efficient units. These efficient firms are identified in the column Efficient DMUs.

4. Application of the PRA procedure to selecting alternative DEA models

As shown in Table 3, different input/output combinations yielded very different results when the firm's efficiencies were analysed. All the units ranked in the first position along the nine ratios displayed in Table 1 were labelled as efficient for the corresponding model that included simultaneously both the considered input and output as variables. The results show that there is a positive correlation between the average efficiency of a model and the number of units labelled as efficient (r = 0.796, P < 0.01). However, the number of selected variables does not correlate either with the number of efficient units labelled or with the average efficiency.

The use of a high number of units labelled as efficient and a high efficiency average are reasonable criteria to reduce the bias of DEA analysis (Cubbin and Tzanidakis, 1998a; Kittelsen, 1999). The number of units identified as efficient should be large enough to form a proper reference set, allow-

ing simultaneously proper discrimination between efficient and inefficient units (Parkin and Hollingsworth, 1997). The PRA procedure allows us detecting the models where a firm is labelled efficient, as shown in Table 4. After this, we could identify the models with a higher number of DMUs labelled as efficient.

Applying the criteria described above, only 10 out of the 49

possible models have an acceptably high minimum number of efficient units, namely ABC12(5), BC123(5), ABC23(4), ABC13(4), AC123(4), AC12(4), BC13(4), BC12(4) and BC23(4) and obviously the GDM ABC123(6). The analysis of these models could help to improve DEA efficiency analysis by identifying both atypical behaviours and factors that determine the unit efficiency. The number of models in this operational set might be further reduced, if the possible efficiency overestimation is taken into account.

The comparison of the individual rankings in each of the nine individual indicators allows the identification of the units

having a special output/input profile. If a particular input and/or output is usually associated with good positions in the

ranking of individual indicators (see Table 1), this variable will be identified as a candidate to be removed. This fact allows us to select DEA models that are significant for the

analysis because they include, or not, the variables that have a special contribution to the efficiency of the firm. The last two columns of Table 4 show these 'contributors' variables for the six efficient firms according to their ranking positions. Additionally, a suggestion is also provided referring to the alternative model suitable for analysing the effects of skipping a particular variable on the efficiency of the unit.



MI Gonzalez-Bravo-Prior-Ratio-Analysis procedure 1219

Table 4 Analysis of the results of the six efficient units under different DEA models

Efficient Minimal Alternative firms Individual ratio (best position) Variables models Crucial models

DMU5 Profit/fixed asset (1/A) 4 = { I (A, B), O(1, 2)) 28 1 ABC23 Value-added/labour (2/B)

DMU11 Profit/total asset (1/C) 2 = {I(C), O(1)} 16 1 ABC23

DMU17 Value-added/tot.asset (2/C) 2= {I(C), 0(2)1 16 2 ABC13 BC13

DMU18 Profit/labour (1/B) 3 = {I(B), 0(1, 3)1 24 B AC123 Sales revenue/labour (3/B) AC12

DMU20 Value-added/fixed asset (2/A) 4 = {I(A, C), 0(2, 3)) 32 A and 3 BC12 Sales revenue/fixed asset (3/A) Sales revenue/total asset (3/C)

DMU29 None best position. Only three second A and 2 BC13 positions.

Column 2 lists the individual ratios for which each firm scores in first position along the nine indicators. Variables identifies the inputs (I) and the outputs (0) that are combined to form the individual ratio for which a given firm scores best. Minimal number of models is the estimated number of models for which a firm will be labelled efficient, along the 49 feasible input/output combinations, according to its best scored positions. This minimal number of models is calculated resolving a combinatorial problem. Column 4 shows the variable/s identified as crucial for the behaviour of a firm. A variable is considered crucial if the individual ranking indicators oscillate between good and poor positions depending on the presence or not of this variable. The column Alternative models shows the model, among those previously selected that has an acceptably high minimum number of efficient units, that could rend a more accurate analysis. These models are the result of removing the variables identified as 'contributors' for a particular DMU.

Finally, seven models can be chosen as an operational subset of the initial 49 models, remaining simultaneously representative and exhaustive: ABC123, ABC12, BC123, ABC23, AC12, BC12 and BC13.

5. The use of alternative models as inputs/outputs combinations based on a PRA improves both the DEA efficiency assessment and the ranking procedures The seven selected models could be analysed resulting in seven efficiency scores. These seven new efficiency measures allow us analysing both the strengths and the weaknesses of the labelled efficient firms by discriminating between them, as shown in Table 5. It is obvious that the efficiency score of certain units strongly depends on the presence of particular variables in the model, as predicted in Table 4. This results in a remarkable drop of the efficiency values, as occurs for DMU18, which falls to 0.13 efficiency in the AC12 model, or for DMU20, dropping to 0.31 efficiency in the model BC12. Thus, the information given in Table 5 gathers the stability of the efficiency scores along the models tested and the importance of a given unit while evaluating the rest allowing ranking the units accurately.

Several methods have been used to rank units in a DEA context (Friedman and Sinuany-Stern, 1997, 1998; Sinuany-Stern and Friedman, 1998; Zhu, 2000; Adler et al, 2002). Table 6 collects a variety of these procedures and shows the remarkable differences between them. The methods listed in the first four columns in Table 6, are the most common in the literature and they are obtained after resolving a single GDM. The first

two methods are based on the confidence in the higher classification of a DMU, as best-performer (Holvad, 2001 www. its.usyd.edu.au/conferences/thredbo/thredbo7/holvad.pdf), if the unit appears repeatedly as evaluator for inefficient firms (Branch-to-benchmark (Avkiran, 1999); well-rounded performer (Chen and Yeh, 1998) or Robustly Efficient Com- parison Set (Brockett et al, 2001)). The Cross-efficiency matrix (CEM) procedure (Doyle and Green, 1994) is a suitable method for discriminating between true efficient units and false positives (Sarkis, 2000), and the Super-efficiency (SUPF) (Andersen and Petersen, 1993) is possibly one of the most utilized ranking methods because of its simplicity. The methods listed in the last four columns in Table 6 are ranking procedures accounting the selected DEA models resulting of input/output combinations with our PRA application described here.

Table 6 shows that the firms achieve very different rank position depending on the ranking methods. Spearman's correlation between the results is only significant in some cases, (ie REF/IMP r = 0.940, P = 0.01; RANK_9/CEM r = 1, P = 0.01; VER_7/CEM r = 0.829, P = 0.05). The no concordance is also supported by means of the non-parametric Kendall-W test (0.494, P = 0.001).

Some procedures are sensitive to particular behaviours of the firms. The ranking derived from the CEM and the one-dimensional analysis represented by

RANK_9, reach

similar results. Both procedures are sensitive to extreme observations, such as firms performing good and poor simultaneously depending on a specific variable. (ie see the true position for DMU18 and DMU20 in Table 6). The use of




Table 5 Results of DEA analysis in the selected models for the efficient firms

BC123 ABC23 ABC12 BC13 AC12 BC12 ABC123

R P R P R P R P R P R P R P

DMU5 19 47.3 22 45.4 19 52.6 8 12.5 1 0 20 65 19 47.4 DMU 11 4 0 0.88 - 3 0 17 23.5 5 20 4 0 3 0 DMU17 22 54.4 22 45.4 20 55 0.87 - 20 100 27 51.8 21 42.8 DMU18 2 0 2 0 0 0 6 0 0.13 - 0 0 2 0 DMU20 16 31.2 15 33.3 1 100 27 81.5 4 50 0.31 - 15 33.3 DMU29 0.97 - 3 33.3 6 50 0.87 - 7 42.8 0.96 - 3 33.3

Seven models capable of resolving the special behaviours of the firms are listed. R: importance of a firm conforming the reference set. This variable accounts for the number of times a firm appears in the reference set for the inefficient firms in a particular model. P: importance of a firm in the reference set. This variable is the number of times a firm is the most important in the reference set for the inefficient firms. That is, the firm achieves the highest weight (A) in the solution. This variable is displayed as a percentage. Shadowed cells show the efficiency ratios for the units not identified as efficient in the corresponding model.

Table 6 Concordance between ranking methods and ranked classifications

DMU REF IMP CEM SUPF RANKS9 AVER_7 EFFA49 AVERA49

DMU5 2 1 1 4 1 1 2 1 DMU11 4 4 4 2 4 2 4 4 DMU17 1 1 2 1 2 3 4 4 DMU18 5 4 6 (31) 5 6 (21) 6 3 5 (8) DMU20 3 2 5 (8) 6 5 (11) 5 1 3 DMU29 4 3 3 3 3 4 5 2

The ranking of the six efficient units in independent analysis with different methods is depicted. REF is the number of times a firm appears in the reference set. IMP is the number of times a firm is the most important in the reference set (ie the highest weight (Q)). CEM and SUPF show the rank position of a firm resulting of cross-efficiency matrix and super-efficiency procedures, respectively. RANKS9 is the variable defined in Table 1.

AVER_7 is the average efficiency score in the seven models selected and analysed. EFF_49 is the number of times a firm is efficient along the 49 models.

AVER_49 is the average efficiency score in the 49 models. Four methods (ie REF, IMP, SUPF and EFF_49) rank only the efficient units and the rank value varies exactly between 1 and 6. When two or more of firms obtain the same result for a method they are ranked in the same position. Other methods allow ranking both efficient and inefficient units. Consequently, some efficient firms could be classified below some inefficient units that occupy positions after the sixth. In this case, the global position, using information about all DMUs, appear in parentheses.

CEM as a procedure to prevent the named false efficiencies (Cubbin and Tzanidakis, 1998b) or self-identifiers (Bauer et al, 1998) is effective but, in our opinion, too radical. It is true that the DMU18 is extremely dependent on Input B (Labour), but this firm is efficient in 50% of models combining the variables set. This firm is neither too good, as measured by the EFF_49, nor too bad, as the RANKS9 and CEM methods show. In this sense, the EFFA49 procedure discriminates the named global efficiencies in favour to partial efficiencies, reflecting the strength of a firm in extreme models (ie a particular individual performance ratio).

The average method represented by AVER_49 detects the influence of a variable when this variable is not included, showing both the overall performance of the firms and their particular behaviours. This ranking method is suitable for the identification of the overestimation and could be taken as an adequate indicator of efficiency. We have studied the

correlation of the efficiency scores for both the efficient and the inefficient units, obtained with the AVER_7, AVER_49 and CEM scores. The results show that there is a significant positive correlation between these three ranking procedures. The

efficiency analysis by means of the proposed PRA provides us with a measure of efficiency (AVER_7) that fits properly with global measures as AVER_49 (r = 0.966) CEM(r = 0.822). These results are besides supported by the Kendall-W test (0.932 and P = 0.000). We believe that the new efficiency measure described here affords a reliable tool for ranking and

discriminating between units.

6. Conclusion

The application of a DEA model that includes simultaneously all the variables selected for the analysis of efficiency can be misleading. Special input/output profiles or special units



MI Gonzalez-Bravo--Prior-Ratio-Analysis procedure 1221

could be misidentified as efficient, resulting in efficiency overestimation. On the other hand, average well-performing units could go unnoticed if they do not show extreme positive behaviours over the rest of the analysed firms. The GDM is usually thought to represent the best possible behaviour of a firm. However, the special performance, the atypical oper- ating input/output structures and the factors determining the unit efficiency can be overlooked if exclusively this model is applied. We have proved that certain firms are repeatedly identified as efficient in a significant number of DEA models despite the fact that they do not show the highest average efficiency score. Their efficiency oscillates between very good and poor behaviours, therefore occupying different positions (into-frontier, near-frontier or far off-frontier) depending on the input/output combinations. In consequence, further surveys are necessary to understand the variations of scores in DEA analysis and their relationship with the behaviour of a firm in an input/output combination.

The resolution of several DEA models, resulting from input/output combinations, is a suitable alternative to reflect the overall behaviour of a firm and the special features af- fecting its performance. This multi-measure approach allows identifying atypical behaviours while providing insights into the factors that determine the unit efficiency. Here, we have showed that a reduced, exhaustive but representative number of models can be easily selected, therefore improving the DEA analysis and allowing ranking units in a accurate way, without losing resolution depth.

Accordingly to the relationship between the ratio analysis and the DEA analysis, a firm that scores best in an output/input ratio will also be efficient in all feasible DEA models that include these combinations. Hence, the PRA procedure is proposed here for detecting the variables that modify the behaviour of firms. This allows choosing some input/output combinations that permit more reliable efficiency measures.

The use of ratio analysis together with DEA methodology could benefit the latter with a major understanding of the factors that determine the unit efficiency and resolving the limitations revealed in the empirical application of this paper. The procedure proposed here yields an efficiency measure that represents the accurate performance of a firm and that fits properly with other measures of overall performance.

References

Adler N, Friedman L and Sinuany-Stern Z (2002). Review of ranking methods in the data envelopment analysis context. Eur J Opl Res 140: 240-265.

Allen R and Thanassoulis E (2004). Improving envelopment in data envelopment analysis. Eur J Opl Res 154: 363-379.

Andersen P and Petersen N (1993). A procedure for ranking efficient units in data envelopment analysis. Mngt Sci 39: 1261-1294.

Avkiran N (1999). An application reference for data envelopment analysis in branch banking: helping the novice researcher. Int J Bank Market 17: 206-220.

Bauer P et al (1998). Consistency conditions for regulatory analysis of financial institutions: a comparison of frontier efficiency methods. J Econ Bus 50: 85-114.

Bowlin W (1998). Measuring performance: an introduction to data envelopment analysis (DEA). J Cost Anal Fall 1998: 3-27.

Brockett P et al (2001). The identification of target firms and functional areas for strategic benchmarking. Eng Econ 46: 274-299.

Charnes A, Cooper WW and Rhodes E (1978). Measuring the efficiency of decision making units. Eur J Opl Res 2: 429-441.

Chen Y and Ali A (2002). Output-input ratio analysis and DEA frontier. Eur J Opl Res 142: 476-479.

Chen T and Yeh T (1998). A study of efficiency evaluation in Taiwan's banks. Int J Serv Ind Mngt 9: 402-415.

Cooper W et al (2001). Sensitivity and stability analysis in DEA: some recent developments. J Prod Anal 15: 217-246.

Cubbin J and Tzanidakis G (1998a). Regression versus data envelopment analysis for efficiency measurement: an application to the England and Wales regulated water industry. Utilities Pol 7: 75-85.

Cubbin J and Tzanidakis G (1998b). Techniques for analysing company performance. Bus Strategy Rev 9: 37-46.

Doyle JR and Green R (1994). Efficiency and cross-efficiency in data envelopment analysis: derivatives, meanings and uses. J Opl Res Soc 45: 567-578.

Dyson R et al (2001). Pitfalls and protocols in DEA. Eur J Opl Res 132: 245-259.

Fernandez-Castro A and Smith P (1994). Towards a general non- parametric model of corporate performance. Omega 22: 237-249.

Friedman L and Sinuany-Stern Z (1997). Scaling units via the canonical correlation analysis in the DEA context. Eur J Opl Res 100: 629-637.

Friedman L and Sinuany-Stern Z (1998). Combining ranking scales and selecting variables in the DEA context. The case of industrial branches. Comput Opl Res 25: 781-791.

Greenberg R and Nunamaker T (1987). A generalized multiple criteria model for control and evaluation of nonprofit organizations. Financial Account Mngt 3: 331-342.

Holvad T (2001). An examination of efficiency level variations for bus services. Paper presented at the Seventh International Conference on Competition and Ownership in Land Passenger Transport (THREDBO 7), Molde, Norway, June 2001.

Jenkins L and Anderson M (2003). A multivariate statistical approach to reducing the number of variables in data envelopment analysis. Eur J Opl Res 147: 51-61.

Kittelsen S (1999). Monte Carlo simulations of DEA efficiency measures and hypothesis tests. Memorandum no. 09/99. Department of Economics, University of Oslo.

Norman M and Stoker B (1991). Data Envelopment Analysis. The Assessment of Performance. Wiley : New York.

Parkin D and Hollingsworth B (1997). Measuring production efficiency of acute hospitals in Scotland, 1991-94: validity issues in data envelopment analysis. Appl Econ 29: 1425-1433.

Sarkis J (2000). An analysis of the operational efficiency of major airports in the United States. J Opns Mngt 18: 335-351.

Serrano-Cinca C and Mar Molinero C (2004). Selecting DEA specifications and ranking units via PCA. J Opl Res Soc 55: 521-528.

Serrano-Cinca C, Fuertes-Calldn Y and Mar Molinero C (2005). Measuring DEA efficiency in Internet companies. Decis Support Syst 38: 557-573.

Sinuany-Stern Z and Friedman L (1998). DEA and the discriminant analysis of ratios to ranking units. Eur J Opl Res 111: 470-478.

Smith P (1990). Data envelopment analysis applied to financial statements. Omega 18: 131-138.




Smith P (1997). Model misspecification in data envelopment analysis. Ann Opns Res 73: 233-252.

Staat M (2001). The effect of sample size on the mean efficiency in DEA: Comment. J Prod Anal 15: 129-137.

Talluri S (2000a). A benchmarking method for business-process reengineering and improvement. Int J Flex Manu Syst 12: 291-304.

Talluri S (2000b). Data envelopment analysis: models and extensions. Decis Line 31: 8-11.

Talluri S and Yoon K (2000). A cone-ratio DEA approach for AMT justification. Int J Prod Econ 66: 119-129.

Thanassoulis E, Boussofiane A and Dyson R (1996). A comparison of data envelopment analysis and ratio analysis as tools for performance assessment. Omega 24: 229-244.

Worthington A (1998). The application of mathematical programming techniques to financial statement analysis. Australian gold production and exploration. Aust J Manage 23: 97-111.

Zhang and Bartels R (1998). The effect of sample size on mean efficiency in DEA with an application to electricity distribution in Australia, Sweden and New Zealand. J Prod Anal 9: 187-204.

Zhu J (2000). Multi-factor performance measure model with an application to Fortune 500 companies. Eur J Opl Res 123: 105-124.

Received December 2004;

accepted May 2006 after two revisions



prior-ratio-analysis procedure to improve data envelopment analysis for performance measurement

Documents