OECD Space Forum:Evaluation / Impact Assessment

Claire Jolly

Head, OECD Space Forum

Science and Technology Policy Division

Directorate for Science, Technology and Innovation

Assessing the Impact of Satellite EO on Society and Policy

Organised by NICT, Keio Univ., Hosei Univ., JAXA

9 November 2015 - Hitotsubashi Hall, Tokyo Japan

The OECD is an international economic organisation.

• A global forum in which governments representatives work together to address the economic, social and environmental challenges of interdependence and globalisation

• A source of economic statistics - provider of comparative data, analysis and forecasts to underpin multilateral co-operation (www.oecd.org)

No vested interest in the space sector... But a recognised expertise in the “economics” of space programmes and activities

2

The Organisation for Economic Co-operation and Development (OECD) - Fast facts

The Organisation for Economic Co-operation and Development (OECD) - Fast facts

• Established: 1961

• Headquarters: Paris

• OECD Centres: Washington, Mexico City, Tokyo, Berlin

• Members: 34

• Secretary-General: Angel Gurría

• Secretariat staff: 2 500

• Nearly 300 expert committees and working groups with participation of +100 countries (all sectors of the global economy covered)

3

The OECD Space Forum

Background: IFP space project (2002-05) led to strong demand to set up an international platform hosted in OECD, complementary to other structures (ITU, UNCOPUOS…).

Missions: Forum established to better identify statistically the sector, and investigate its economic dimensions as an infrastructure for the larger economy.

• Development of comparable statistics and indicators, exchange lessons learned and best practices (industry associations, NSOs…)

• Assess the contribution of space technologies to addressing major challenges the world faces (e.g. environment) including socio-economic impacts

4

Participation in the OECD Space Forum

5

The Space Forum activities are managed by the OECD Secretariat, with analytical inputs from the Forum’s Steering Group.

The Steering Group is composed of 10 space–related administrations (often a department dedicated to strategy) and/or ministries with oversight of their country’s space activities, from OECD countries:– Canadian Space Agency (Canada) – CNES (France)– DLR (Germany)– ASI (Italy) – NASA (US) – Korean space agency (Korea)– Norwegian Space Centre (Norway) – Swiss Space Office (Switzerland)– UK Space Agency (United Kingdom)– And the European Space Agency (ESA)

In addition close co-operation with industry associations, the private sector, other agencies and ministries…

The Space Forum contributes to different OECD missions

Selected Science Technoloy and Innovation (STI) Themes 2015-16

Stats and indicators

Impacts

Innovation Policy Country Reviews X X

The Impact of Public Support for Business R&D and innovation

X X

Innovation, Inclusiveness and Development

X X

Higher Education Institutions in the Knowledge Triangle

X X

6

As well as cooperation with different directorates and divisions (Environment, Trade, Agriculture, Fisheries…)

Highlighting the economic significance of space activities

Bringing together experts and building a community of practitioners

Making progress on key methodological issues in indicators and statistics

Making progress on evaluation / impact assessments of space investments

7

Achievements so far

It starts with the numbers

OECD Measures, Analyses and Compares

• Comparable data across time and countries

• International methods

• Thorough research

8

Examining socio-economic impacts

OECD International Futures Programme 9

Selected economic impacts in different sectors

10Source: OECD (2014), The Space Economy at a Glance, OECD Publishing, Paris

Value of monitoring sea routes with satellites

Source: OECD (2014), The Space Economy at a Glance, OECD Publishing, Paris

12

Growing demand (justification for spending, more complex programmes, internationalisation of funding mechanisms and expected spillovers)

Evaluation culture and practices uneven across countries, and in some case across national administrations

Fuzziness of basic concepts (difficulty to transpose)

Mismatch between manuals/guidebooks vs. studies’ daily realities (e.g. availability of primary data)

Ignorance about general evaluation guidelines

Need to be aware of unreasonable expectations from “customers”, with ensuing unreasonable methodological approaches to meet these expectations.

Some pointers on evaluation and impact assessments (1/2)

13

The best impact assessments on space applications:

• The earlier the better when launching assessments, to create a “history” of case studies and time series

• Higher quality, when technology / scientific impacts are also provided

(not only focus on economic & social)

• Survey method or peer review, rather than black box method with too many assumptions

• Open tender process for assessments (competition pays off)

Some pointers on evaluation and impact assessments (2/2)

14

On assessments of space investments: more effort is needed in building the knowledge base internationally to provide:

Know-how and valid experiences to practitioners (avoiding reinventing the wheel)

Evidence-based information to decision-makers and citizens on benefits (and limitations) of space applications

OECD Space Forum contributions to the field: Bringing together communities of practices (e.g. dedicated expert workshops,

IAF Space Economy Technical Committee, SEB group with GEO, …) Delivering outputs to facilitate international discussions and transfers of

know-how (Handbook on measuring the space economy, forthcoming OECD working papers…)

The way forward….

Workshop Objectives:Define foundation case studies for demonstration and comparison of methodologies for assessment of geospatial information and a path forward for research. Identify best practice methodologies for the processes from data to decision.

Societal benefit focus areas:Disaster Mitigation and Response – focusing on natural disasters, and extreme eventsEcosystems based Management – highlighting the land-water-energy nexus

For details, see www.socioeconomicbenefits.org 

“Data to Decisions: Valuing the Societal Benefit of Geospatial Information”.

A workshop organized by the Socioeconomic benefits community of practice, in collaboration with OECD, NASA and USGS

March 10-11, 2016, OECD, Paris, France

Thank you for your attention.

16