sdmx in the unido statistical production processssis_bsc – only metadata at level data set and at...

SDMX in the UNIDO Statistical Production Process

8-9 March 2010 1OECD/UNECE Expert Group on SDMX, Geneva Valentin Todorov


Valentin TodorovUNIDO

[email protected]

OECD/UNECE Expert Group on SDMX (Geneva 8-9 March 2010)

mailto:[email protected]



Outline

• Introduction and Overview• UNIDO statistical production process and GSBPM• To start with: consuming the OECD.Stat Web Service• Exchanging data and metadata with SDMX-ML• Collaborative software development• Summary and Outlook



About UNIDO

• UNIDO was set up in 1966 • Became a specialized agency of the UN in 1985• Promote industrialization throughout the developing world• 173 Member States (as of 11 December 2008)• Headquarters in Vienna• Represented in 35 developing countries



About Statistics in UNIDO

• Service Module “Industrial Governance and Statistics”:– monitor, benchmark and analyse the industrial performance and

capabilities– formulate, implement and monitor strategies, policies and

programmes to improve the contribution of industry to productivity growth and the achievement of the UN Millennium Development Goals (MDGs)

• Building capabilities in industrial statistics - providing technical assistance to:– Introduce best practice statistical methodologies and software

systems – Enhance the quality and consistency of the industrial statistics

databases

http://www.unido.org/doc/5066

http://www.un.org/millenniumgoals/

http://www.un.org/millenniumgoals/





Management Processes

Quality Management

INDSTATData

Collection

INDSTATData

Transformation

INDSTATData

Dissemination

INDSTATData

Analysis

INDSTAT Metadata Management

Sales and marketing

User support (stat.info) Software developmentMethodology development

Support Processes



A.2INDSTAT

data and metadatacollection

A.1Structural Metadata

Maintenance

B.1INDSTAT

Data and metadataTransformation

NSONSO

NSO

OECD

Othersources

UNSD

Othersources

B.2Index of industrial

production

WorldBank

Othersources

B.3MVA/GDP

D.1Yearbookproduction

D.2INDSTAT 2

CD production

D.3INDSTAT 4

CD production

D.4IDSB

CD production

D.5WEB: Country brief production

D.6UNDATA

IMF

B.4CPI and

Exchange rates

UNSD

B.5Population

COM-TRADE

B.6Trade data

Othersources

SQLData &

Metadata



A.2INDSTAT



Maintenance

B.1INDSTAT


NSONSO

NSO

OECD

Othersources

UNSD

Othersources


production

WorldBank

Othersources

B.3MVA/GDP


D.2INDSTAT 2

CD production

D.3INDSTAT 4

CD production

D.4IDSB

CD production


D.6UNDATA

IMF

B.4CPI and

Exchange rates

UNSD

B.5Population

COM-TRADE

B.6Trade data

Othersources

SQLData &

Metadata



A.2INDSTAT



Maintenance

B.1INDSTAT


NSONSO

NSO

OECD

Othersources

UNSD

Othersources


production

WorldBank

Othersources

B.3MVA/GDP


D.2INDSTAT 2

CD production

D.3INDSTAT 4

CD production

D.4IDSB

CD production


D.6UNDATA

IMF

B.4CPI and

Exchange rates

UNSD

B.5Population

COM-TRADE

B.6Trade data

Othersources

SDMX-ML

SQLData &

Metadata



Data cubes (the data we are interested in)

• Data set: SSIS_BSC – Structural Business Statistics (Business by size class)

• Time series - annual data - all available years • Dimensions:

– COU - Country: AUS, AUT, BEL, DEU, etc.– SRC - Source: BSC - Business by Size Class – SCL - Size class: Total – ISIC3 – ISIC Revision 3 - all ISICs at 2-, 3- and 4-digit level.



Data cube summary (cont)

• Dimensions (cont):– VAR - Variable:

• ENTR - Number of enterprises • ESTB - Number of establishments • WASE - Wages & Salaries of employees • EMPE - Number of employees • EMPN - Total employment (number engaged) • EMPF - Number of female employees • VAFC - Value added at factor costs • VALU - Value added • PROD - Production • GITG - Gross investment



OECD Import – the process



OECD Import – Code Mapping

• Variables • Countries



OECD Import – INDSTAT database



Web Services• Are fundamental building blocks of distributed computing

over the Internet• Provide the means for software to connect to other

software applications• Use ubiquitos web protocols and formats: HTTP, XML,

SOAP WSDL• Service provider and service consumer are

implementation independent

• Keep the stove pipes apart• SDMX benefits from having a common

information model: we can base WS on it



OECD.Stat Web Service• Gives access to .Stat data

• (Almost) fully SDMX 2.0 compliant• Some limitations apply:

– Only annual, bi-annual, etc. data OK– Each data set has own DSD OK– Each query defines a separate data cube OK

• Other features– Streaming, compression– time stamps, incremental updates– metadata structure definitions and reference metadata



OECD.Stat WS functions used (Data)

• Authenticate– returns a session ID used for all subsequent WS operations– the same session ID can be reused for several SDMX requests

within a certain timeout– without authentication – retrieval of limited data set possible

• GetDataStructureDefinition– to retrieve the Data Structure Definition, Code lists and Concepts

related to the data set SSIS_BSC

• GetGenericData– to retrieve data in generic format from the data set SSIS_BSC– specify country and a list of variables in the query– time series - annual data for all available years



Data Structure Definition Query Example

• Select data set: SSIS-BSC

<message:Query> <KeyFamilyWhere>

<Or> <KeyFamily>SSIS_BSC</KeyFamily>

</Or> </KeyFamilyWhere>

</message:Query>



Generic Data Query Example

• Select data set SSIS-BSC• Select country: Austria• Select variables:

– ENTR, ESTB, WASE, etc.• Wild card – select all ISIC

codes by omitting the dimension

• Set time format - annual• Set time interval



Client Application (Data)



Reference Metadata



OECD.Stat functions used (Reference Metadata)

• Authenticate• GetMetadata

– to retrieve reference metadata in generic format from the data set SSIS_BSC

– only metadata at level data set and at level country are available– specify country (dimension COU)



Metadata query: dimension member

• Select data set: SSIS-BSC• Query a specific dimension: country

<MetadataWhere> <And>

<DataSet>SSIS_BSC</DataSet> <Dimension id = "COU">AUT</Dimension>

</And> </MetadataWhere>



Client Application (Reference Metadata)



Collaborative development

• A low budget, volunteer project • A team spread through multiple time zones and

geographical locations• Collaboration tools for virtual teams:

– A Wiki was set up at wikispaces.com, to build SDMX knowledge, to specify the basic requirements and the desired output: • Publicly available at http://sdmx.wikispaces.com/

– CloudShare - to share copies of the IT environment - for Demos, Proofs-of-Concept, Training, or other applications

– Google Docs at http://docs.google.com for collaborative editing of documents, presentations, etc.

http://sdmx.wikispaces.com/

http://docs.google.com/



Collaborative development: Wiki



Collaborative development: CloudShare



A.2INDSTAT



Maintenance

B.1INDSTAT


NSONSO

NSO

OECD

Othersources

UNSD

Othersources


production

WorldBank

Othersources

B.3MVA/GDP


D.2INDSTAT 2

CD production

D.3INDSTAT 4

CD production

D.4IDSB

CD production


D.6UNDATA

IMF

B.4CPI and

Exchange rates

UNSD

B.5Population

COM-TRADE

B.6Trade data

Othersources

SDMX-ML

SDMX-ML

SDMX-ML

SDMX-ML

SQLData &

Metadata

SDMX-ML

SDMX-ML

SDMX-ML

SDMX-ML

SDMX-ML

SDMX-ML



THE END



Examples



Questionnaire



OECD Questionnaire

sdmx in the unido statistical production processssis_bsc – only metadata at level data set and at...

Documents