The Future of Social Statistics

Martine DurandOECD Chief Statistician

Seminar on Shifting ParadigmsNew York, 18 February, 2011

Overview

– Where have we come from?

– How has the OECD responded?

– The current environment : where are we now?

– Which set of indicators?

– Challenges ahead

2

Where have we come from?“The focus of social indicators is to measure levels of living,

and social and economic factors considered to influence levels of living”, UN Handbook on Social Indicators, 1989

“A sound system of social statistics is vital to the effective development of social policy, to informed decision-making on policy issues, and to the evaluation of the impact of social and economic policy”, UNSC Expert Group, 1996

“There is a need for more internationally agreed statistical standards and guidelines, and for improved frameworks in which to summarize policy outcomes”, UNSC Expert Group, 1996

3

Where have we come from?Social statistics and indicators are about...

both a discrete set of domains (demography, employment, health, education, social and family policies, welfare systems, physical security, etc...)

and a specific way of looking at these and other (economic) domains, focusing on differences in conditions among population groups and on distributions rather than on aggregates only household income is an economic statistic income gap between men and women is (also) a social statistic the distribution of household income is (also) a social statistic

4

Where have we come from?

5

Outcomes(skills, poverty, health status)

Outputs(beneficiaries, take-up rates,

benefit levels, graduation rates)

Inputs and processes(spending, programme features)

Context(demography, socio-economic characteristics, labour

market)

... Social statistics also combine different perspectives...

Where have we come from?

basic demographic and contextual statistics are the first things any government collects...

– Oldest surviving Census data comes from China (Han Dynasty, 2AD )

– The Doomsday Book (commissioned by William the Conqueror in 1086)

6

... Importantly, social statistics are about PEOPLE

Where have we come from?

And outcome measurement has a long history, predating that of official economic statistics

“Life and Labour of the People” (1889); statistical analysis of poverty in London, Charles Booth

“Recent Social Trends in the US” (1933); two-volume report released by the Hoover Committee on Social Trends, chaired by Wesley C. Mitchell (NBER)

7

Where have we come from?

8

Statistical map ofWhitechapel, London, 1886

Red is upper socio-economic areas

Black is “semi-criminal” areas

Source: Charles Booth, 1889

Where have we come from?

Despite long-standing interest in social statistics ...

...Still widespread perception that social statistics are ‘soft’ as compared to ‘hard’ economic statistics because they lack an additive and integrated accounting structure such as the SNA

9

Where have we come from?

Following the development of the SNA, there have been several attempts to develop frameworks for social statistics:

– System of Social and Demographic Statistics, UNSD

– Social Indicators Movement

10

Where have we come from? System of Social and Demographic Statistics

(1975) prepared by Richard Stone for UNSD

– Ambition: to develop a ‘system’ for social statistics matching the SNA

– How: to create an accounting framework for social statistics focusing on flows between different states

– Objective: measures of inputs, outputs, and outcomes in a single framework

11

Where have we come from?

The attempt failed due to:

– Differences among countries in state of development of social statistics and national priorities

– No agreement on common concepts, definitions and measurement instruments

– Lack of a single common metric for integration and aggregation of fundamentally different domains

– Cost of collecting some of the data

– Lack of clear policy relevance for a social accounting framework

12

Where have we come from?

The Social Indicators Movement in the 1970s

– strong push towards measuring social outcomes in a more systematic way (e.g. social contacts, health status, time use)

– did not attempt to produce an over-arching accounting framework for all social activities

13

Where have we come from?

In the 1970s, the OECD also had a significant work programme around social indicators: – Measuring Social Well-Being: A Progress Report on

the Development of Social Indicators, OECD, 1976.

Social Accounting Matrices – introduced extra detail for specific socio-economic

issues into the national accounts framework

14

Where have we come from?

These attempts also failed to produce a stable set of social statistics

– despite best intentions, many indicators were still input-based or of too limited scope with no overall framework for guiding social policy and tracking the impact of social programmes

– the key policy challenges of the 1980s were largely seen to be economic rather than social in nature and social indicators were therefore viewed as a ‘luxury’ rather than a necessity

15

Where have we come from?

Important lessons to be learned : – crucial to articulate what is being measured and why to

be useful for policy decision-making

– limited applicability of accounting and integrated frameworks to social statistics

16

Where have we come from ?Integrated framework for economic statistics

Strong linkages between economic statistics and economic policy issues (e.g. labour, prices, national accounts, government finance and balance of payments) allowing economic policy monitoring

Single (additive) metric

Integrated accounts

Framework for social statistics

Some policy domains are interconnected (e.g. employment , gender gaps, poverty), but not all. Difficult to monitor “overall” social policy

No single metric (and controversies on valuation for non-market activities)

Minimum set of subject-matter statistics and indicators 17

Where have we come from?

Important lessons to be learned : – crucial to articulate what is being measured and why to

be useful for policy decision-making

– limited applicability of accounting and integrated frameworks to social statistics

– frameworks for social statistics need to be sufficiently flexible to be adapted to countries with different social concerns

better to develop an analytical rather than an accounting framework

18

How has the OECD responded?

Since the late 1980s, the OECD’s response has therefore been pragmatic and policy focused:

– Developing indicators linked to clear policy objectives in specific domains

19

How has the OECD responded?

In several areas, the OECD has developed a reliable reporting system based on comparable concepts and definitions, e.g.:– Social expenditure database (SOCX)– Health Data, System of Health Accounts– PISA, Education Indicators – Family Database

Working in close collaboration with national experts (INES, Health data correspondents)

Covering contextual, input, output and outcome indicators

20

How has the OECD responded?

Social expenditures consistent reporting of public and private social expenditures in key domains

21

Public and private social expenditure as percentage of GDP in 2007

How has the OECD responded?

Health care quality indicators: comparison of health systems performance across a number of key dimensions

22

How has the OECD responded?PISA: measures of standardised performance in reading science and math among students aged 15

23

Year 2006Ratio between mean scores of students in the top and bottom decile of the distribution of reading scores

1.0

1.1

1.2

1.3

1.4

1.5

1.6

1.7

1.8

1.9

2.0

How has the OECD responded?

Family Database: an on-line database of family outcomes and policies

24

The current environment Renewed interest in measuring and monitoring

outcomes in a more encompassing/less discrete way: – International initiatives (MDGs, HDI, OECD Measuring Progress,

Stiglitz report and follow-up; UNECE/OECD/Eurostat Task Force on Sustainable Development)

– National initiatives (e.g. State of the USA, Measuring Australia’s Progress, UK National Well-Being, etc)

– Go beyond social field (economic; environmental)

In contrast to the Social Indicators Movement, most current initiatives have a coherent analytical framework indicating what is being measured and why

25

The current environment Interest in measuring social outcomes is driven by:

– citizens’ and governments’ strong demand for more information on what really matters to people beyond economic aggregates

– evidence-based policy and the use of measurable policy targets for evaluations and impact assessments

– increased ability to measure key outcomes that were regarded as inherently un-measurable only a few years ago

– cross-country comparisons of performance as populations are tied more closely together through globalisation and governments wish to understand better the roles of institutions and drivers

26

Where are we now?

Is the progress/well-being measurement agenda a new paradigm for social statistics?

Can it represent the new unifying framework for social statistics?

27

Where are we now?

28

Measuring progress: an encompassing framework focusing on people’s well-being and outcomes

GDP

Materialwell-being

People’s overall well-being

Regrettables

Capital stocksPhysical, natural, human, social

Where are we now?

Similar to, but broader in scope than, the Social Indicators Movement :

– Not only what happens in society from the perspective of governments, but also what happens in governments from the perspective of citizens

– Not only conditions of people but also those of the environment where people live (exposure to environmental hazards, pollution, housing) as they affect their well-being

– Not only people’s objective conditions but also subjective feelings about them

29

Where are we now?

It DOES NOT provide a comprehensive unifying accounting framework for all social statistics as Richard Stone attempted in the 1970s…

…but it DOES provide a clear conceptual framework for outcomes indicators

30

Which set of indicators?

What outcome indicators are we talking about? – measures that are relevant for a broad range of

societies but can be ‘declined’ in different ways to reflect national concerns

– measures that are relevant for designing evidence-based policies

31

– a tool for social monitoring and reporting (needed for periodic medical check-up to assess the patient’s condition)

Which set of indicators ?

The answer is NO :– Progress measures : the entire car dashboard (e.g.

indicators of speed, gasoline left, oil pressure, road conditions), i.e. continuous reporting based on a comprehensive set of measures

32

Are these measures in competition with other initiatives (MDGs, EU2020, targets in national planning documents, e.g. India)?

– MDGs : focus more on “red light” indicators on the dashboard signalling ‘critical conditions’ (e.g. lack of gasoline) and need for immediate action

Which set of indicators?

What needs to be done? Recommendations from the Stiglitz-Sen-Fitoussi Commission :

– Extend the measurements of economic resources, putting more emphasis on the resources of households and their distribution

– Develop measurements of subjective well-being and quality of life, as well as of inequalities in their distribution

– Produce information that allows better assessments of the sustainability of growth, [and develop indicators of environmental impact of economic activities]

33

What is the OECD doing?

During OECD 50th Anniversary (“better policies for better lives”), publication of “How’s Life?” report

Accompanied by a number of methodological projects done under the auspices of CSTAT, in collaboration with other international organisations

Aim is to improve existing indicators and build new ones

34

What is the OECD doing?

“How’s Life?” will present existing comparable indicators for OECD and selected emerging countries (the “low hanging fruits”)

“How is Life?” draws on the SSF recommendations– primary focus is on measuring people’s well-being

– Material conditions– Quality of life

– secondary focus is on measuring sustainability– Key capital stocks

Work in progress: will evolve over time

35

What is the OECD doing?

Material well-being

– Income and wealth

– Jobs and earnings

– Housing

Quality of life

– Health status

– Education and skills

– Work and life balance

– Civic engagement and governance

– Social connections

– Environmental quality

– Personal security

– Subjective well-being

36

Well-Being Dimensions

What is the OECD doing? Choice of indicators under each dimension based

on common set of criteria:– Validity/Relevance (of the available measures w.r.t. the

outcomes of interest?; are they amenable to policy change?)

– Cross-country comparability of information

– Depth (information on distribution of outcomes)

– Well-established sources (is the information from official/reliable sources?)

– Recurrence of data collection

Indicators assessed against each of these criteria: adequate (√), limited (~) and missing (X)

37

What is the OECD doing?

38

Challenges ahead

Work required to build a common/minimum set of headline indicators

Need to fill in statistical gaps, but:

– National statistical systems are under pressure to deliver progress/well-being indicators at the same time as NSOs’ budgets are being squeezed and priorities in social fields conflict with other domains

39

Challenges ahead

40

Developing relevant indicators in new or under-developed domains

– In some domains indicators are not available or only provide averages or only cover specific at-risk populations

– Need to build new valid indicators based on common definitions and concepts; important to establish priorities and timeframes

Way forward: UNECE /Eurostat/OECD TF

work on SD; INSEE/Eurostat Sponsorship; UN Regional Commissions; launch new research projects ?

Challenges ahead The case of subjective indicators

– For some aspects of well-being both perceptions and reality matter for analysis and policies; need for objective indicators and information on values that people attach to various dimensions

– Most NSOs already collect subjective information in some of their surveys (General Social Survey, Time Use Survey), but this information is often not comprehensive and not comparable internationally

Way forward: OECD Guidelines for developing SWB indicators in collaboration with Eurostat/EU SILC, EQLS, UK, US 41

Challenges ahead

Official v.s. non-official sources

– In some domains, the only type of comparative information that currently exists is through non-official sources

Way forward:

• Expand the scope of official instruments: Household Surveys; LFS; Time Use Surveys; EU SILC

• Use non-official sources if NSOs judge that data are of sufficient quality and cheaper; work with (semi) private providers (e.g. EQLS) ?

42

Challenges ahead Depth and Consistency : use of micro-data and

common standards and definitions– Matching of various survey-based and administrative data

to assess outcomes for population subgroups, and to allow analysis of causality and policy impacts

– Make micro data available to research community– Differences in underlying concepts and definitions limit

integration at country-level and cross-country comparability

Way forward: OECD expert groups on joint distribution of income, consumption and wealth; on integrating inequalities in NA (with Eurostat); on micro-data access

43

Challenges ahead

44

The challenges just outlined require serious methodological work in a number of areas

A number of IOs and countries have started new projects that will address some of them

Those need to be shared and coordinated to ensure maximum coherence and consistency in standards and definitions across instruments used to collect indicators

Conclusions– Past international attempts to design a fully

integrated framework for social statistics have failed– More pragmatic approach restricted to specific

domains has had some success both at country and international level

– Today, there is renewed and strong interest for a more encompassing framework focusing on outcomes

– Measuring Progress and SSF follow-up represent a great opportunity to move ahead

– Not without challenges, but these can be addressed

45