sdmx-edi and sdmx-ml
DESCRIPTION
How implement a "data sharing model" and a pull mode using SDMX standardsTRANSCRIPT
IPA 2007 - Tirana - INSTAT
SDMX-EDISDMX-ML
SDMX-EDISDMX-ML
Skopje 15 Jan 2009
Format that was used during starting period for external trade data
Later incorporated into Intrastat and Extrastat systemsThere were utilized different media of transmission and
passing data files:– Tapes– CD-ROM– CDs– Diskettes– Files – by post
First standard defined slowly for different domains in statistics– GESMES/DSIS – still used for PRODCOM
– GESMES/CB – Gesmes Central Banks
– GESMES/TS – Gesmes Time Series
– Version 3.0 has been used as background to next generation standard – SDMX-EDI
• Developed as continuation of the GESMES/TS Standard
• Improved in the Version SDMX 2.0
SDMX
SDMX - Statistical Data and Metadata eXchange.
•Initiative of seven international organizations to build Standard for Data and Metadata eXchange
•Using modern information technology, with an emphasis on aggregated data.
•All definitions and tools are available on Internet:www.sdmx.org
The need for a standard…
• Enhance electronic data and metadata exchange
• Promote interoperability between different systems
• Improve the quality of transmitted data (Timeliness & Punctuality, Accessibility & Clarity, Accuracy, Comparability)
SDMX
SDMX is primarily focused on the exchange and dissemination of statistical data and metadata.
We have normally two different approach to exchange data: PUSH and PULL
SDMX
PUSH mode means that the data provider takes action to send the data to the party collecting the data.
PULL mode implies that the data provider makes the data available via the Internet. The data consumer then fetches the data on his own initiative.
SDMX
SDMX promotes a “data sharing” model to facilitate low-cost, high-quality statistical data and metadata exchange.
Data Providers publishes the availability of data/metadata to Data Consumers and the latter are responsible for fetching the data/metadata at will.
.
Data-Sharing Exchange
YOU
NSI
1
2
3
Notes About Data-Sharing• Data-sharing only works if there are standard
formats
• Like the Web itself, a data-sharing model relies on pull exchanges, not push exchanges– Data consumers discover the data they need, and its location, and then
go and get it
– Data producers don’t have to send data
An easy way to understand the SDMX-IM
10369
– What is the subject of the measurement?– What units does it measure in?– What country or geographical region, if any, does it
apply to?– When was the measurement made?
An easy way to understand the SDMX-IM
Behind each of these questions is a particular idea, or "concept", which is used to describe the data.
– the Subject is "total population"– the Unit of measure is "thousands of people"– the Country is “Germany"– the Time is "1 January 2001"
An easy way to understand the SDMX-IM
• We now know that our number represents the fact that the total population of Country “Germany” on 1 January, 2001, was 10,369,000.
An easy way to understand the SDMX-IM
Data Set
Data Set: Structure
Computers need structure of data
•Concepts and terms
•Code lists
•Data values
•How these fit together
Unit Multiplier
Unit
Topic
Time/Frequency
CountryStock/FlowData Set: Structure
TopicA Brady Bonds
B Bank Loans
C Debt Securities
CountryAR Argentina
MX Mexico
SA South Africa
Stock/Flow1 Stock
2 Flow
ConceptsTOPIC
COUNTRY
FLOW
Structural Definitions
16457
SA,B,1,1999-06-30=16547
Data Makes Sense
Metadata
Structural metadata must be associated with the data, otherwise it becomes impossible to identify, retrieve and browse the data.
Structures in the SDMX-IM
Structure Components
Concept Scheme Concept
Code List Code
Category Scheme Category
Data Structure Definition (DSD) • Dimensions
• Attributes
• Measures
• Groups
• The simplest explanation of a data structure definition is that it is a set of descriptor concepts, associated with a set of data, which allow us to understand what that data means.
Data Structure Definition
DSD components
• Dimension (e.g. frequency, reference area):
• Attribute (e.g. title, observation status):
• Measure (e.g. turnover index, # of births, # of deaths):
• Groups:– Grouping of dimensions in order to attach group attributes
(e.g. sibling group)
Data Structure Definition
…………..………. Structure
…………..... ComponentList
……………………….... Component
cd Data Structure Definition
DSD
KeyDescriptorAttributeDescriptor GroupKeyDescriptorMeasureDescriptor
DataAttribute Measure Dimension
MeasureType Dimension
Dataflow Definition Item Scheme::Category
Item Scheme::Concept
XSmeasure
DataSet
ConceptRole
+ role: ConceptRoleType
Item Scheme::Code
1..*
1
0..1
1
0..*
1
1
1
0..*
1
0..*
1
1
1
0..*
1
0..*
1
0..*
0..*
0..*
1
0..*
1
0..* 0..*1
defines
0..*
1..*
1
0..*
0..*
0..*
0..*
0..*
0..*
SDMX V1 and V2
Year MonthTurnover
index Status Confidentiality2002 January 84.5 actual free2002 February 85.6 actual free2002 March 95.4 actual free2002 April 106.2 actual free2002 May 98.0 actual free2002 June 95.3 actual free2002 July 105.4 actual free2002 August 107.1 actual free
2002 September 105.2 actual free2002 October 109.4 actual free2002 November 104.5 actual free2002 December 111.9 actual free2003 January 89.1 provisional free2003 February 88.3 provisional free2003 March 96.1 provisional free
Source: National Statistical Service of GreeceData prepared to be transmitted to the European Commission (including EUROSTAT)
Table 1. Deflated turnover index (on volume of sales) for retail trade for Greece (no adjustment). Reference period: January 2002 to March 2003.
(monthly data - Base year: 2000)
EXAMPLE
DATASET1
STS Sample Dataset
Year MonthTurnover
index Status Confidentiality
2002 January 84.5 actual free
2002 February 85.6 actual free2002 March 95.4 actual free2002 April 106.2 actual free2002 May 98.0 actual free2002 June 95.3 actual free2002 July 105.4 actual free2002 August 107.1 actual free2002 September 105.2 actual free2002 October 109.4 actual free2002 November 104.5 actual free2002 December 111.9 actual free2003 January 89.1 provisional free
2003 February 88.3 provisional free2003 March 96.1 provisional free
Source: National Statistical Service of GreeceData prepared to be transmitted to the European Commission (including EUROSTAT)
Table 1. Deflated turnover index (on volume of sales) for retail trade for Greece (no adjustment). Reference period: January 2002 to March 2003.
(monthly data - Base year: 2000)
STS Sample Dataset
Dimensions
Measure
Attributes
Dimensions
STS DSD componentsDataflow: STSRTD_TURN_M
Concept Concept ID Code List Valuereference period TIME_PERIOD Month/Yearreporting country REF_AREA CL_AREA_EE EL - Greecebase year STS_BASE_YEAR CL_STS_BASE_YEAR 2000type of index STS_INDICATOR CL_STS_INDICATOR TOVV - Turnover deflated (volume of sales)activity STS_ACTIVITY CL_STS_ACTIVITY Retail tradeadjustment ADJUSTMENT CL_ADJUSTMENT No (Neither seasonally or working day adjusted)frequency FREQ CL_FREQ monthly datatitle TITLE Title of the exchanged datasetstatus OBS_STATUS CL_OBS_STATUS actual/provisional dataconfidentiality OBS_CONF CL_OBS_CONF Free (free of publication data)decimals DECIMALS CL_DECIMALS 1 - One
Measures Turnover index OBS_VALUE observations
Groups Time series Set of ordered monthly data (01/02-12/02)
Dimensions
Attributes
Male Female
131 35 16624057 23871 4792829400 28345 577454799 4187 8986
2572350 2683230 52555802562077 2674534 5236611
17423319
10837 10581 214186038 6331 12369
Number1338329283
Rate1.8
Years82.3 75.5 78.3
Data prepared to be transmitted to the European Commission (including EUROSTAT)
Life expectancy at birth
Number of persons
Marriages
Total fertility rate
ImmigrantsEmigrants
Divorces
Population on 01/01/2006Population on 01/01/2005Deaths under 1 yearBirths outside marriage
Statistical adjustmentDeathsBirthsNet migration
SexTotal
Demographic Characteristic
Demography Rapid Questionnaire_Table RQFI05V1. Data for Finland. Reference period: January to December 2005 (annual,
provisional data - 1st revision).
EXAMPLE
DATASET2
Demography Sample Dataset
Male FemaleNumber of persons
Statistical adjustment 131 35 166
Deaths 24057 23871 47928
Births 29400 28345 57745
Net migration 4799 4187 8986
Population on 01/01/2006 2572350 2683230 5255580
Population on 01/01/2005 2562077 2674534 5236611
Deaths under 1 year 174
Births outside marriage 23319
Immigrants 10837 10581 21418
Emigrants 6038 6331 12369Number
Divorces 13383
Marriages 29283Rate
Total fertility rate 1.8Years
Life expectancy at birth 82.3 75.5 78.3
Data prepared to be transmitted to the European Commission (including EUROSTAT)
Demography Rapid Questionnaire_Table RQFI05V1. Data for Finland. Reference period: January to December 2005 (annual provisional
data - 1st revision).
Demographic Characteristic
Sex
Total
Demography SampleM
ea
su
res
Dimensions
Attributes
Demography DSD components
Concept Concept ID Codel List Valuesreference period TIME_PERIOD 01-2005 to 12-2005reporting country COUNTRY CL_COUNTRY Finlandsex SEX CL_SEX male / femaledeomographic characteristic DEMO CL_DEMO # of births, # of deaths etc.frequency FREQ CL_FREQ annual datatitle TITLE Title of the exchanged datasetstatus OBS_STATUS CL_OBS_STATUS provisional datareference table TAB_NUM RQFI05V1version REV_NUM 1st revisionStaistical adjustment ADJT number of personsdeaths DEATHST number of personsbirths LBIRTHST number of personsnet migration NETMT number of personspopulation on 01/01/06 PJAN1T number of personspopulation on 01/01/05 PJANT number of personsdeaths under 1 year DEATHUN1 number of personsbirths outside marriage LBIRTHOUT number of personsimmigrants IMMIT number of personsemigrants EMIGT number of personsdivorces DIV pure numbermarriages MAR pure number
total fertility rate TFRNSI decimal indexlife expectancy at birth LEXPNSIT number of years
Groups SectionSet of annual demographic characteristics from FI (01/05-12/05)
Measures
Dimensions
Attributes
Dataflow: DEMOGRAPHY_RQ
Data Set Identifier Variables Form DescriptionSTSIND_PROD (_M, _Q) 110 I Production in industry
STSIND_TURN (_M, _Q) 120, 121, 122 N, ITurnover in industry, total, domestic and non-domestic (total, Euro-zone, non-Euro-zone)
STSIND_ORD (_M, _Q) 130, 131, 132 N, INew orders received in industry, total, domestic and non-domestic (total, Euro-zone, non-Euro-zone)
STSIND_EMPL (_M, _Q) 210 N, I Number of persons employed, Number of employees, in industry
STSIND_HOUR (_M, _Q) 220 N, I Hours worked in industry
STSIND_EARN (_M, _Q) 230 N, I Gross wages and salaries in industry
STSIND_PRIC (_M, _Q) 310, 311, 312, 340 IOutput prices in industry, total, domestic market, non-domestic market (total, Euro-zone, non Euro-zone), import prices (total, Euro-zone, non-Euro-zone)
STSCONS_PROD(_M, _Q)
110, 115, 116 I Production in construction, total, building construction, civil engineering
STSCONS_ORD (_M, _Q) 130, 135, 136 N, INew orders received in construction, total, building construction and civil engineering
STSCONS_EMPL(_M, _Q)
210, 211 N, I Number of persons employed, Number of employees, in construction
STSCONS_HOUR(_M, _Q)
220 N, I Hours worked in construction
STSCONS_EARN(_M, _Q)
230 N, I Gross wages and salaries in construction
STSCONS_PRIC (_M, _Q) 310, 320, 321, 322 I Output prices in construction, construction costs, material costs, labour costs
STSCONS_PERM(_M, _Q)
411, 412 N, I Building permits, number of dwellings or square metres of useful floor area
STSRTD_TURN (_M, _Q) 120, 123 N, I Turnover in retail trade, value or deflated
STSRTD_EMPL (_M, _Q) 210, 211 N, I Number of persons employed, Number of employees, in retail trade
STSSERV_TURN(_M, _Q)
120, 123 N, I Turnover in repair and other services, value or deflated
STSSERV_PRIC (_M, _Q) 310 I Outut prices in other services
STSSERV_EMPL(_M, _Q) 210, 211 N, INumber of persons employed, Number of employees, in repair and other services
STSSERV_CAR (_M, _Q) Number of car registrations STSOTHER_OTH(_M, _Q)
Any other indicator not mentioned in the list aboveIPA 2007 - Tirana - INSTAT
Concept Mnemonic Concept Name Format Description Code list
ADJUSTMENT Adjustment AN1Code defining the adjustment of data such as working day or seasonally adjusted, etc.
CL_ADJUSTMENT
FREQ Frequency AN1Frequency of the series (e.g. A, Q, M).
CL_FREQ
OBS_CONF Confidentiality flag AN1Confidentiality status of the observation
CL_OBS_CONF
OBS_PRE_BREAK Pre-break observation value AN…15Observation value if the reason of the "break" did not show up.
[Conditional]
OBS_STATUS Status flag AN1Status of the observation, such as normal, estimated or provisional
CL_OBS_STATUS
OBS_VALUE Value AN…15 The value of the index.
ORGANISATION Organisation AN3
Reporting/sending or receiving organisation used in the message administration section.
CL_ORGANISATION
REF_AREA Reference area AN2
Reporting Country in ISO code (The country, or geographical/political group of countries that the measured economic phenomenon relates to)
CL_AREA_EE
STS_ACTIVITY Economic Activity code AN6NACE Rev. 1.1 & special STS aggregates
CL_STS_ACTIVITY
STS_BASE_YEARSeries variation in short-term stats context
AN4Concept to distinguish series variations in a short-term stats context
CL_STS_BASE_YEAR
STS_INDICATOR STS Indicator AN4Type of indicator, such as production, turnover, etc.
CL_STS_INDICATOR
STS_INSTITUTIONInstitution originating STS dataflow
AN1Institution originating STS dataflow
CL_STS_INSTITUTION
TIME_FORMAT Time Format Code AN3 Technical use in message.TIME_PERIOD Time Period AN…35 The time period of the data.
IPA 2007 - Tirana - INSTAT
Code List Mnemonic Code List Name Format
CL_ADJUSTMENT Adjustment code AN1
CL_AREA_EE Country code AN2
CL_FREQ Frequency code AN1
CL_OBS_CONF Confidentiality flag AN1
CL_OBS_STATUS Observation status flag AN1
CL_ORGANISATION Organisation code list AN3
CL_STS_ACTIVITY STS Economic Activity code list AN6
CL_STS_BASE_YEAR Suffix in short-term stats context code list AN4
CL_STS_INDICATOR Indicators index code AN4
CL_STS_INSTITUTION Institution originating STS dataflow code list AN1
IPA 2007 - Tirana - INSTAT
Value Description VariablePROD Production 110, 115, 116TOVT Turnover (total turnover, non-deflated) 120TOVD Turnover, domestic market (non-deflated) 121TOVE Turnover, non-domestic market (non-deflated) 122TOVV Turnover deflated (volume of sales) 123TOVX Turnover, non-domestic market (non-deflated) (non-Euro-zone) 122TOVZ Turnover, non-domestic market (non-deflated) (Euro-zone) 122DEFL Deflator of sales 330ORDT New orders received (total) 130, 135, 136ORDD New orders received, domestic market 131ORDE New orders received, non-domestic market 132ORDX New orders received, non-domestic market (non-Euro-zone) 132PRON Output prices for industry and services (total) 310PRIN Output prices, domestic market 311PREN Output prices, non-domestic market
(can be approximated by unit value index , variable 313)312, 313
PREX Output prices, non-domestic market (non-Euro-zone) 312PREZ Output prices, non-domestic market (Euro-zone) 312IMPR Import prices (total) 340IMPX Import prices (non-Euro-zone) 340IMPZ Import prices (Euro-zone) 340EMPL Number of persons employed
(can be approximated by number of employees, variable 211)
210, 211
HOWK Hours worked 220WAGE Gross wages and salaries 230PNUM Building permits, number of dwellings 411PSQM Building permits: square metres of useful floor area 412CSTI Construction costs (total) 320CSTM Construction costs, material costs 321CSTL Construction costs, labour costs 322
CSTO Output prices for construction
(approximation for construction costs, variable 320)
310
CREG Car registrations (not in STS Regulation)
IPA 2007 - Tirana - INSTAT
SDMX-IM
From SDMX-IM to messages
SDMX-EDI SDMX-ML
• Based on a common Information Model– SDMX-EDI (GESMES/TS)
• EDIFACT syntax• Time series oriented – One format for Data Sets
– SDMX-ML• XML syntax• Four different formats for Data Sets• Easier validation (XML based)
From SDMX-IM to messages
UNA:+.? 'UNB+UNOC:3+EUROSTAT+Unknown+060627:0000+IREF000001++GESMES/TS++++1'UNH+MREF000001+GESMES:2:1:E6'BGM+74'NAD+Z02+EUROSTATNAD+MR+Unknown'NAD+MS+EUROSTAT'CTA+CC+:V. Patruno'DSI+SODI_IPI_PROD_M'STS+3+7'DTM+242:200601010000:203'DTM+Z02:200501200503:710'IDE+5+STS'GIS+AR3'GIS+1:::-'ARR++M:GR:W:PROD:NS0020:1:2000:200501200503:710:111.11:A:F:+222.22:A:F:+333.33:A:F:+444.44:A:F:+555.55:A:F:+666.66:A:F:+777.77:A:F:+888.88:A:F:+99.99:A:F:+123.45:A:F:+212.21:A:F:+234.56:A:F:'FNS+Attributes:10'REL+Z01+4'ARR+7+M:GR:W:PROD:NS0020:1:2000'IDE+Z10+COLLECTION'CDV+A'IDE+Z10+AVAILABILITY'CDV+A'UNT+26+MREF000001'UNZ+1+IREF000001'
SDMX-EDI
SDMX-ML
XML format for the exchange of SDMX-structured data and metadata.
<?xml version="1.0" encoding="UTF-8"?><!-- Created with SDMX Converter v2.1 --><CompactData
xmlns="http://www.SDMX.org/resources/SDMXML/schemas/v2_0/message"xmlns:sts="urn:sdmx:org.sdmx.infomodel.keyfamily.KeyFamily=ESTAT:STS:compact"xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"xsi:schemaLocation="http://www.SDMX.org/resources/SDMXML/schemas/v2_0/message SDMXMessage.xsdurn:sdmx:org.sdmx.infomodel.keyfamily.KeyFamily=ESTAT:STS:compact ESTAT_STS_Compact.xsd"><Header>
<ID>SODI_IPI_PROD_M</ID><Test>true</Test><Name xml:lang="en">SDMX Tutorial Message</Name><Prepared>2006-06-27T00:00:00</Prepared><Sender id="EUROSTAT">
<Contact><Name xml:lang="en">V. Patruno</Name><Department xml:lang="en">IT Dept</Department><Role xml:lang="en">Maintainer</Role><Email>[email protected]</Email>
</Contact></Sender>
<DataSetAgency>EUSTAT</DataSetAgency><DataSetID>SODI_IPI_PROD_M_02</DataSetID><DataSetAction>Append</DataSetAction><Extracted>2006-01-01T00:00:00</Extracted><ReportingBegin>2005-01-01T00:00:00</ReportingBegin><ReportingEnd>2005-03-31T00:00:00</ReportingEnd>
</Header><sts:DataSet>
<sts:SiblingGroup REF_AREA="GR" ADJUSTMENT="W" STS_INDICATOR="PROD" STS_ACTIVITY="NS0020" STS_INSTITUTION="1" STS_BASE_YEAR="2000" UNIT="PC" UNIT_MULT="0" DECIMALS="2" TITLE_COMPL="Elements of the full national etc."/>
<sts:Series FREQ="M" REF_AREA="GR" ADJUSTMENT="W" STS_INDICATOR="PROD" STS_ACTIVITY="NS0020" STS_INSTITUTION="1" STS_BASE_YEAR="2000" COLLECTION="A" AVAILABILITY="A" TIME_FORMAT="P1M">
<sts:Obs TIME_PERIOD="2005-01" OBS_VALUE="111.11" OBS_STATUS="A" OBS_CONF="F"/><sts:Obs TIME_PERIOD="2005-02" OBS_VALUE="222.22" OBS_STATUS="A" OBS_CONF="F"/><sts:Obs TIME_PERIOD="2005-03" OBS_VALUE="333.33" OBS_STATUS="A" OBS_CONF="F"/><sts:Obs TIME_PERIOD="2005-04" OBS_VALUE="444.44" OBS_STATUS="A" OBS_CONF="F"/><sts:Obs TIME_PERIOD="2005-05" OBS_VALUE="555.55" OBS_STATUS="A" OBS_CONF="F"/><sts:Obs TIME_PERIOD="2005-06" OBS_VALUE="666.66" OBS_STATUS="A" OBS_CONF="F"/><sts:Obs TIME_PERIOD="2005-07" OBS_VALUE="777.77" OBS_STATUS="A" OBS_CONF="F"/><sts:Obs TIME_PERIOD="2005-08" OBS_VALUE="888.88" OBS_STATUS="A" OBS_CONF="F"/><sts:Obs TIME_PERIOD="2005-09" OBS_VALUE="99.99" OBS_STATUS="A" OBS_CONF="F"/><sts:Obs TIME_PERIOD="2005-10" OBS_VALUE="123.45" OBS_STATUS="A" OBS_CONF="F"/><sts:Obs TIME_PERIOD="2005-11" OBS_VALUE="212.21" OBS_STATUS="A" OBS_CONF="F"/><sts:Obs TIME_PERIOD="2005-12" OBS_VALUE="234.56" OBS_STATUS="A" OBS_CONF="F"/>
</sts:Series></sts:DataSet></CompactData>
ExAMplE
The first part of the message contains information about the XML namespaces and schemas used
ExAMplE
The next part of the message is the header, which identifies the particular data being sent in this case, including the sender and receiver.
ExAMplE
The third part of the message contains the data values, or “observations” in SDMX terminology.
Name of message Short description Schema file
1 Structure Definition Message
Contains a data structure definition Fixed
2 Generic Data Message Conveys data in a form independent of a data structure definition. It is designed for data provision on websites and in any scenario where applications receiving the data may not have detailed understanding of the data set's structure before they obtain the data set itself.
Fixed
3 Compact Data Message Exchange of large data sets in a data structure definition-dependent form
Derived from data structure definition message
4 Utility Data Message For schema-based functions, such as validation, in a data structure definition-dependent form
Derived from data structure definition message
5 Cross-sectional Data Message
Exchange of many observation types in a data structure definition-dependent form
Derived from data structure definition message
6 Query message To query a database to obtain an SDMX-ML message as the result
Fixed
SDMX-ML: Six standard messages
Cross-Sectional Data Set<demo:DataSet REV_NUM="1" TAB_NUM="RQFI05V1"> <demo:Group COUNTRY="FI" FREQ="A" TIME="2005" TIME_FORMAT="P1Y"> <demo:Section DECI="0" UNIT="PERS" UNIT_MULT="0"> <demo:ADJT OBS_STATUS="P" SEX="F" value="35"/> <demo:DEATHST OBS_STATUS="P" SEX="F" value="23871"/> <demo:LBIRTHST OBS_STATUS="P" SEX="F" value="28345"/> <demo:NETMT OBS_STATUS="P" SEX="F" value="4187"/> <demo:PJAN1T OBS_STATUS="P" SEX="F" value="2683230"/> <demo:PJANT OBS_STATUS="P" SEX="F" value="2674534"/> <demo:ADJT OBS_STATUS="P" SEX="M" value="131"/> <demo:DEATHST OBS_STATUS="P" SEX="M" value="24057"/> <demo:LBIRTHST OBS_STATUS="P" SEX="M" value="29400"/> <demo:NETMT OBS_STATUS="P" SEX="M" value="4799"/> <demo:PJAN1T OBS_STATUS="P" SEX="M" value="2572350"/> <demo:PJANT OBS_STATUS="P" SEX="M" value="2562077"/> <demo:ADJT OBS_STATUS="P" SEX="T" value="166"/> <demo:DEATHST OBS_STATUS="P" SEX="T" value="47928"/> <demo:LBIRTHST OBS_STATUS="P" SEX="T" value="57745"/> <demo:NETMT OBS_STATUS="P" SEX="T" value="8986"/> <demo:PJAN1T OBS_STATUS="P" SEX="T" value="5255580"/> <demo:PJANT OBS_STATUS="P" SEX="T" value="5236611"/> </demo:Section> <demo:Section DECI="0" UNIT="PURE_NUMB" UNIT_MULT="0"> <demo:DIV OBS_STATUS="P" SEX="T" value="13383"/> <demo:MAR OBS_STATUS="P" SEX="T" value="29283"/> </demo:Section> <demo:Section DECI="3" UNIT="PURE_NUMB" UNIT_MULT="0"> <demo:TFRNSI SEX="T" value="1800"/> </demo:Section> </demo:Group></demo:DataSet>
SDMX-ML “Model-Driven” XML
Approach
What Do You Need to Do?
If you want to use SDMX to exchange data, what do you need to do?
• Step 1: Define a data structure definition (key family)• Step 2: Create XML Schemas describing data formats• Step 3: Use schemas in implementation
– Create data– Exchange data– Import data– Create deliverables (CSV, HTML, etc.)
SDMX Registry• A repository for keeping
– Structural metadata (e.g. CodeLists, ConceptSchemes, DSDs, MSDs)
– Provisioning information (e.g. Dataflows, Provision Agreements)
• Repository is accessible via a Web Service accepting SDMX-ML messages
• GUI for user interaction over the Web• Authentication using CIRCA accounts• DSW – “standalone” Java GUI
https://webgate.ec.europa.eu/sdmxregistry/
SDMX Registry
Query Message
• This is another special-purpose message. It is used to convey a query to a database which then returns an SDMX-ML message
• For web services and database-driven applications
Query Message
Query SDMX
DataSDMX-
ML
RSS
WSNSI
sodi.istat.itcon.istat.it
Query SDMX
Dati(SDMX-
ML)
RSS
WS
demo.istat.it
DB
WS
demogr
DEMO
SODI RSS
script
sodi.istat.itcon.istat.it
Query SDMX
Dati(SDMX-
ML)
RSS
WS
demogr
Thank you for your attention
Vincenzo Patruno: [email protected]