composite indicators for the measurement of economic performance p. roberti f. oropallo istat...

Composite Indicators for the Measurement of Economic Performance

P. RobertiF. Oropallo

ISTAT

Productivity, Competitiveness and the New Information EconomyBusiness , Systemic and Measurement Issues

NESIS FP5 ISTAT – RomeJune 26, 2003

June, 26 2003 Composite Indicators for the Measurement of Economic Performance - Roberti / Oropallo

2

We live with too many Indicators

LISBON OBJECTIVES “to become the most

competitive and dynamic knowledge-based economy in the world, capable of sustainable economic growth ...and social cohesion”

A host of indicators that have been proposed to monitor this goal.

They can be grouped in five areas:

General economic background Employment Economic Reform Social Cohesion Environment


3

Indicators can beone-dimensional ormulti-dimensionalComposite indicators have received increasing attention in recent years.Various methodologies have been developed to handle aggregation and related problems: Aggregation systems Deciding on the phenomenon to be measured Selection of sub-indicators Assessing the quality of the data Assessing the relationships between the sub-indicators Testing for Robustness and Sensitivity


4

State of the Art –Methodological IssuesA number of methodologies can be applied for the development of composite indicators. They include: Multiple linear regression models Principal components analysis and factor analysis Cronbach alpha Neutralization of correlation effect Efficiency frontier Distance to targets Experts opinion (budget allocation) Public opinion Analytic Hierarchy Process

JRC – EC (2002) - State-of-the-art Report on Current Methodologies and Practices for Composite Indicator Development – Joint research Centre – European Commission - Institute for the Protection and Security of the Citizen Technological and Economic Risk Management I-21020 Ispra (VA) Italy - Prepared by the Applied Statistics Group June 2002


5

Is there a clear Framework yet?

How can a framework be developed?

By defining an analytical framework with precise properties!


6

Property of Indicators

• Micro founded• Scope fulfilling • Purpose oriented• Well- behaved• Consistent• Decomposable• Multidimensional - Composite


7

MICRO FOUNDED

that is, based on a comprehensive database that embraces all aspects of enterprise features (ad hoc surveys can cover some aspects) Integration of different data sources of micro data Quality test of the integration process (matching procedures / estimation)

In the second part of this presentation examples of the opportunities opened up by an Integrated Database are shown (Diecofis Project www.istat.it/diecofis Year of reference is 1998-2000)

Sources are: (1) Structural Business Statistics (2) Administrative data (Foreign Trade, Commercial account, Fiscal and Social security data)

http://www.istat.it/diecofis


8

SCOPE FULFILLING Indicators can measure size, change and dispersion Changes in an Indicator’s value can have different causes and

lead to different conclusions depending on underlying combinations

Socio-economic phenomena have different and complex dimensions.

Different indicators can serve different purposes, i.e. measure:- heterogeneity/dispersion -performance (moving toward the mean)

- overall systemic performance ß-performance (generalised move upward/downward)

overall comparative performance (e.g. catching up/lagging behind)

stratification (to evaluate differences in systemic structures and whether they represent a “stratum” - as with Yitzhaki’s decomposition)


9

PURPOSE ORIENTED

Socio-economic phenomena may have many facets and change can result from a combination of different patterns

Appropriate indicators may be needed in different circumstances


10

WELL-BEHAVED AND CONSISTENT

Indicators inconsistency may arise for different reasons

Reference to condition of - Lorenz dominance (focusing on relative

differences) - Pareto superiority (focusing on levels) - stochastic dominance (focusing on both

dimensions)


11

DECOMPOSABLE & MULTIDIMENSIONAL

To be able to study patterns take into consideration more than one

dimension/aspects


12

Developments in poverty analysis are a good example of the possible problems and avenues to solve them

Number of people Income gaps Welfare dimensions Multidimensional aspects

Headcounts Gaps FGT indices (squared gaps, etc.) Multidimensional indices


13

Developing a similar framework is important for the analysis of systemic performance and the benchmarking of “economic textures”

Since many factors and forces are at work to determine, condition and produce different outcomes.

The quote that follows can serve to grasp the problem and possible approach to address it


14

“The fact that many of the smaller EU economies do either better or worse than the larger ones is partly due to larger EU economies contributing more to the overall EU mean than smaller economies, which means that they are less able to diverge from the mean. A second explanation is due to structural conditions. The industrial distribution of small economies is often concentrated in a few sectors, while larger economies are more diverse. This can shift the scores towards the mean for many indicators in large economies, while small economies can exhibit either a high or low innovative capacity, depending on the sectors that dominate the economy. Of course, this shift towards high or low technology sectors is not accidental, but reflects both public and private institutions seeking out areas of comparative advantage and high profitability”.

(EU Commission)


15

The “analytical framework” and indicators that can serve to study and benchmark different levels/dimensions of “systemic features” Systemic maps (whole jigsaw) The overall aggregate picture: EU level The national, regional, local picture The sectional picture The occupational picture Systemic strength and weakness, at a point in time (cross-section

analysis) and overtime (longitudinal analysis) Map transitions: features, patterns and evolution (“New” vs.

“Old”) Systemic change and its features, at the aggregate/disaggregate

levels


16

Decomposable indicator how

The GINI index measures the concentration of a particular phenomenon (0=no concentration, 100=maximum concentration). It can be divided into three elements:

overlapbetweenwithinGINI

LppyypGGINIK

k

K

ikikik

K

kkkk

)(11


17

Distributions and Overlap When overlap in the decomposition is high, it is very hard to judge which group is the best/worst, because distributions cross

Mean (A) ≈ Mean (B) ≈ Mean (C) performance

N


18

Distributions and Overlap

When overlap in the decomposition is low it is easy to determine which group is the best/worst performer

Mean (A) Mean (B) Mean (C) performance

N


19

Mono-dimensional Analysis of performance (Overall)

(GINI index calculated on exports)


20

Mono-dimensional analysis of performance (Exports) (By NACE Sectors)


21

Multi-dimensionalanalysis of performance

Decomposition for each dimensiondddd LbwG

Decomposition of the between component for each dimension across K classes

K

k

dk

d bb1


22

To make composite indicators easy to use and interpret

Requires Normalization

Requires Aggregation

mdbMax

bbn dk

dkd

k ,..1)(

ddd

m

d

dkd

wandwwith

bnwm

C

110

11


23

Benchmarking performance levels or gaps?

How far away from best performers enterprises are?

)()(

100

:)(

100

100

dk

dk

dkd

k

dk

dkd

k

dk

dk

bMaxbbMax

Gap

isThatbMax

bGap

bnGap


24

Weighted Composite Gap

How far away from best performers enterprises are?

ddd

m

d

dkd

wandwwith

Gapwm

Cgap

110

11


25

Composite Indicator (Breakdown by NACE sector) Three dimensions One Dimension

Sector Value Added Sector Employment Sector Export Sector CompositeI 100 I 100 DK 100 I 78 E 62 DJ 74 DM 69 DK 75 DK 55 DK 70 DG 49 DJ 55 DF 53 DB 63 DL 46 DM 48 DJ 49 DL 47 DB 43 DB 44 DG 41 DM 43 DJ 43 DL 43 DL 34 DA 39 I 33 DG 41 DM 33 DE 35 DC 33 DA 31 DA 31 DG 32 DN 26 E 29 DB 27 H 28 DA 23 DH 22 DE 22 DH 26 G 22 DF 22 K 22 DI 25 DH 20 DE 21 DI 20 E 24 DI 16 DC 21 DH 20 DN 23 DF 9 DI 21 DN 10 F 22 DE 7 DN 20 DC 9 DC 21 K 2 G 10 C 9 DD 6 DD 2 H 9 G 6 DF 4 F 1 K 9 F 3 K 4 E 0 F 9 DD 1 C 4 C 0 C 4 H - G 2 H 0 DD 3


26

NACE sectors

ACTIVITY DESCRIPTION C PRODUCTS FROM MINING AND QUARRYING DA FOOD PRODUCTS, BEVERAGES AND TOBACCO DB TEXTILES AND CLOTHING INDUSTRY PRODUCTS DC LEATHER AND LEATHER PRODUCTS DD WOOD AND PRODUCTS OF WOOD AND CORK (EXCEPT FURNITURE) DE PULP, PAPER AND PAPER PRODUCTS; RECORDED MEDIA; PRINTING SERVICES DF COKE, REFINED PETROLEUM PRODUCTS AND NUCLEAR FUEL DG CHEMICALS, CHEMICAL PRODUCTS AND MAN-MADE FIBRES DH RUBBER AND PLASTIC PRODUCTS DI OTHER NON METALLIC MINERAL PRODUCTS DJ BASIC METALS AND FABRICATED METAL PRODUCTS DK MACHINERY AND EQUIPMENT N.E.C. DL ELECTRICAL AND OPTICAL EQUIPMENT DL30 MACHINERY AND COMPUTERS DL313 Insulated wire and cable DL32 RADIO, TELEVISION AND COMMUNICATION EQUIPMENT AND APPARATUS DL332 Instruments and appliances for measuring, checking, testing, navigating DL333 Industrial process control equipment DM TRANSPORT EQUIPMENT DN OTHER MANUFACTURED GOODS N.E.C.

E ELECTRICAL ENERGY, GAS, STEAM AND WATER F CONSTRUCTION WORK G WHOLESALE AND RETAIL TRADE SERVICES G5143 Wholesale trade services of electrical household appliances and radio and television goods G5164 Wholesale trade services of machinery and computer G5165 Wholesale trade services of Industrial equipment H HOTEL AND RESTAURANT SERVICES I TRANSPORT, STORAGE AND COMMUNICATION SERVICES I642 Telecommunications services J FINANCIAL INTERMEDIATION SERVICES K REAL ESTATE, RENTING AND BUSINESS SERVICES K7133 Renting services of office machinery and equipment including computers K72 COMPUTER AND RELATED SERVICES M - N -O EDUCATION – HEALTH AND SOCIAL - OTHER SOCIAL SERVICES

The Sectors in italic are defined “ICT sectors” by OECD


27

Composite Indicator (Breakdown by NACE sector) Three dimensions of enterprises’ performance: (1)Value Added (2)Employment (3)Exports

a) one dimension-three areas

-

10

20

30

40

50

60

70

80

90

100

DK I DM DJ DG DL DB DA DC DN DH E DF DI DE G K F H C DD

VALUE ADDED

EMPLOYMENT

EXPORT


28

Composite Indicator (Breakdown by NACE sector)

COMPOSITE

0

10

20

30

40

50

60

70

80

90

100

I DK DJ DM DB DL DG DA E DH DF DE DC DI DN G H K F C DD

b) 3 dimensions into one indicator

Short fall from top or average


29

Composite Indicator (Breakdown by regions)

-

20

40

60

80

100

LOM LAZ PIE VEN EMR TOS FVG TAA MAR LIG ABR SAR PUG UMB CAM VDA BAS SIC MOL CAL

VALUE ADDED

EMPLOYMENT

EXPORT

Three dimensions of enterprises’ performance: (1)Value Added (2)Employment (3)Exports

a) 3 mono-dimensional indicators


30

-

20

40

60

80

100

LOM LAZ PIE VEN EMR TOS FVG TAA MAR LIG ABR SAR PUG UMB CAM VDA BAS SIC MOL CAL

Composite Indicator

Composite Indicator (Breakdown by regions)

b) from 3 to one 3-dimensional indicator

Short fall


31

Composite Indicator (by size of firm) Three dimensions of enterprises’ performance: (1)Value

Added (2)Employment (3)Exports

-

20

40

60

80

100

>99 10-49 50-99 1-2 3-9

VALUE ADDED

EMPLOYMENT

EXPORT

COMPOSITE

-

20

40

60

80

100

>99 10-49 50-99 1-2 3-9

Short fall


32

Conclusions

The analysis which has been presented draws from research results

under two related FP5 projects DIECOFIS and NESIS that deal with

different mapping and benchmarking aspects.

A strong investment in the design and development of a complex and

wide ranging system of enterprise micro data which have been

integrated and systematised into one single “hub”.

The analysis is founded on micro-data drawn from the integrated and

systematised enterprise SIS, which gives high flexibility and

allows to aggregate and disaggregate indicators a la carte.


33

Thank You

composite indicators for the measurement of economic performance p. roberti f. oropallo istat...

Documents