the use of administrative sources for statistical purposes frameworks for the access and use of...
TRANSCRIPT
![Page 1: The Use of Administrative Sources for Statistical Purposes Frameworks for the Access and Use of Administrative Data](https://reader031.vdocuments.mx/reader031/viewer/2022032705/56649da25503460f94a8e4b7/html5/thumbnails/1.jpg)
The Use of Administrative Sources for Statistical Purposes
Frameworks for theAccess and Use ofAdministrative Data
![Page 2: The Use of Administrative Sources for Statistical Purposes Frameworks for the Access and Use of Administrative Data](https://reader031.vdocuments.mx/reader031/viewer/2022032705/56649da25503460f94a8e4b7/html5/thumbnails/2.jpg)
LegalFrameworks
![Page 3: The Use of Administrative Sources for Statistical Purposes Frameworks for the Access and Use of Administrative Data](https://reader031.vdocuments.mx/reader031/viewer/2022032705/56649da25503460f94a8e4b7/html5/thumbnails/3.jpg)
Data Sources
There are two main sources of data
1. Statistical surveys
2. Administrative records
Legal frameworks are necessary toenable and control the flow of datafrom both sources.
![Page 4: The Use of Administrative Sources for Statistical Purposes Frameworks for the Access and Use of Administrative Data](https://reader031.vdocuments.mx/reader031/viewer/2022032705/56649da25503460f94a8e4b7/html5/thumbnails/4.jpg)
National Legal Frameworks
• Most statistical institutions have legislation defining their roles and responsibilities, e.g. a ‘Statistical Law’
• Many new statistical laws introduced in the last 10 - 15 years
• Some existing statistical laws have been revised to include provisions for the use of administrative data
![Page 5: The Use of Administrative Sources for Statistical Purposes Frameworks for the Access and Use of Administrative Data](https://reader031.vdocuments.mx/reader031/viewer/2022032705/56649da25503460f94a8e4b7/html5/thumbnails/5.jpg)
• Some national legal frameworks give more powers than others for access to administrative data
• Historical, political and cultural factors will have an impact on national frameworks, therefore these are not harmonised between countries
National Legal Frameworks
![Page 6: The Use of Administrative Sources for Statistical Purposes Frameworks for the Access and Use of Administrative Data](https://reader031.vdocuments.mx/reader031/viewer/2022032705/56649da25503460f94a8e4b7/html5/thumbnails/6.jpg)
International Frameworks
• The European Union is developing a legal framework for official statistics, including references to the use of administrative data
![Page 7: The Use of Administrative Sources for Statistical Purposes Frameworks for the Access and Use of Administrative Data](https://reader031.vdocuments.mx/reader031/viewer/2022032705/56649da25503460f94a8e4b7/html5/thumbnails/7.jpg)
Case Study - IrelandStatistics Act 1993:30. - (1) For the purpose of assisting the
Office in the exercise of its functions under this Act, the Director General may … request any public authority to—
(a) allow officers of statistics … to have access to … any records in its charge, and
(b) provide the Office … with copies or extracts from any such record,
and the public authority shall … comply with any such request free of charge.
![Page 8: The Use of Administrative Sources for Statistical Purposes Frameworks for the Access and Use of Administrative Data](https://reader031.vdocuments.mx/reader031/viewer/2022032705/56649da25503460f94a8e4b7/html5/thumbnails/8.jpg)
31. - (2) If any public authority proposes to introduce, revise or extend any system for the storage and retrieval of information … it shall consult with the Director General and accept any recommendations that he may reasonably make …
(3) The Director General or any public authority may request the National Statistics Board to arbitrate on and, when agreement cannot be reached, to make recommendations to the [Prime Minister] for his decision on proposals made by the Director General …
![Page 9: The Use of Administrative Sources for Statistical Purposes Frameworks for the Access and Use of Administrative Data](https://reader031.vdocuments.mx/reader031/viewer/2022032705/56649da25503460f94a8e4b7/html5/thumbnails/9.jpg)
Stages of Development
The Eurostat “Handbook of GoodPractices” identifies three key stagesin the development of a legalframework:
• Foundation
• Consolidation
• Evolution
![Page 10: The Use of Administrative Sources for Statistical Purposes Frameworks for the Access and Use of Administrative Data](https://reader031.vdocuments.mx/reader031/viewer/2022032705/56649da25503460f94a8e4b7/html5/thumbnails/10.jpg)
Stages of Development
• Foundation stage - The statistical institute starts to get access to administrative sources
• Consolidation stage – Administrative data are used to build registers and produce statistics
![Page 11: The Use of Administrative Sources for Statistical Purposes Frameworks for the Access and Use of Administrative Data](https://reader031.vdocuments.mx/reader031/viewer/2022032705/56649da25503460f94a8e4b7/html5/thumbnails/11.jpg)
Stages of Development
• Evolution stage - The legal framework encourages greaterco-operation between administrative data suppliers and the statistical institute concerning the collection of data for multiple purposes
![Page 12: The Use of Administrative Sources for Statistical Purposes Frameworks for the Access and Use of Administrative Data](https://reader031.vdocuments.mx/reader031/viewer/2022032705/56649da25503460f94a8e4b7/html5/thumbnails/12.jpg)
How Data Collection is Changing
0
10
20
30
40
50
60
70
80
1950 1975 2000 2025
Administrative
Postal
Interview
![Page 13: The Use of Administrative Sources for Statistical Purposes Frameworks for the Access and Use of Administrative Data](https://reader031.vdocuments.mx/reader031/viewer/2022032705/56649da25503460f94a8e4b7/html5/thumbnails/13.jpg)
Timing• National statistical institutes are at
different stages in this process
• Different areas of statistics are at different stages in this process
• It takes at least 30 years to move from a population census based on paper questionnaires to one fully based on administrative data
![Page 14: The Use of Administrative Sources for Statistical Purposes Frameworks for the Access and Use of Administrative Data](https://reader031.vdocuments.mx/reader031/viewer/2022032705/56649da25503460f94a8e4b7/html5/thumbnails/14.jpg)
Why is This Relevant?
• Legal frameworks should reflect the desired future position rather than the current reality, otherwise they will quickly become a barrier
• Start planning now for the next revision to the legal framework
![Page 15: The Use of Administrative Sources for Statistical Purposes Frameworks for the Access and Use of Administrative Data](https://reader031.vdocuments.mx/reader031/viewer/2022032705/56649da25503460f94a8e4b7/html5/thumbnails/15.jpg)
Legislation on the Internet
• Austria - www.statistik.at/_institution/bstg2003.pdf
• Ireland - www.irishstatutebook.ie/ZZA21Y1993.html
– See sections 30 and 31
• Norway - www.ssb.no/english/about_ssb/statlaw/statlov_en.html
– See chapter 3-2
• Germany - www.destatis.de/download/ueber/bstag03_e.pdf
– See article 8
![Page 16: The Use of Administrative Sources for Statistical Purposes Frameworks for the Access and Use of Administrative Data](https://reader031.vdocuments.mx/reader031/viewer/2022032705/56649da25503460f94a8e4b7/html5/thumbnails/16.jpg)
PolicyFrameworks
![Page 17: The Use of Administrative Sources for Statistical Purposes Frameworks for the Access and Use of Administrative Data](https://reader031.vdocuments.mx/reader031/viewer/2022032705/56649da25503460f94a8e4b7/html5/thumbnails/17.jpg)
Policy Frameworks
• Government policy on data sharing– How can the national statistical
institute influence this?
• Codes of practice– International
– National
![Page 18: The Use of Administrative Sources for Statistical Purposes Frameworks for the Access and Use of Administrative Data](https://reader031.vdocuments.mx/reader031/viewer/2022032705/56649da25503460f94a8e4b7/html5/thumbnails/18.jpg)
International Codes of Practice
• United Nations Fundamental Principles of Official Statistics– Principle 5: Data for statistical purposes
may be drawn from all types of sources, be they statistical surveys or administrative records. Statistical agencies are to choose the source with regard to quality, timeliness, costs and the burden on respondents
![Page 19: The Use of Administrative Sources for Statistical Purposes Frameworks for the Access and Use of Administrative Data](https://reader031.vdocuments.mx/reader031/viewer/2022032705/56649da25503460f94a8e4b7/html5/thumbnails/19.jpg)
International Codes of Practice• European Statistical System Code of
Practice– Principle 2: Statistical authorities must
have a clear legal mandate to collect information for European statistical purposes. Administrations, enterprises and households, and the public at large may be compelled by law to allow access to or deliver data for European statistical purposes at the request of statistical authorities
![Page 20: The Use of Administrative Sources for Statistical Purposes Frameworks for the Access and Use of Administrative Data](https://reader031.vdocuments.mx/reader031/viewer/2022032705/56649da25503460f94a8e4b7/html5/thumbnails/20.jpg)
National Codes of Practice
• Can reassure the public that data will only be used for specific and reasonable purposes
• Should be made available to the public, e.g. via the internet site of the national statistical institute.
![Page 21: The Use of Administrative Sources for Statistical Purposes Frameworks for the Access and Use of Administrative Data](https://reader031.vdocuments.mx/reader031/viewer/2022032705/56649da25503460f94a8e4b7/html5/thumbnails/21.jpg)
OrganisationalFrameworks
![Page 22: The Use of Administrative Sources for Statistical Purposes Frameworks for the Access and Use of Administrative Data](https://reader031.vdocuments.mx/reader031/viewer/2022032705/56649da25503460f94a8e4b7/html5/thumbnails/22.jpg)
Using Administrative Sources in Practice
The use of administrative sources is enabled by a legal framework, in the context of a policy framework
But - these are not usually detailed enough to cover all the administrative arrangements for access and use
![Page 23: The Use of Administrative Sources for Statistical Purposes Frameworks for the Access and Use of Administrative Data](https://reader031.vdocuments.mx/reader031/viewer/2022032705/56649da25503460f94a8e4b7/html5/thumbnails/23.jpg)
What Else is Needed?
• Some sort of agreement:- Service Level Agreement (SLA)- Administrative Protocol- Contract- Informal or verbal agreement- Other type of agreement according to national customs and practices
![Page 24: The Use of Administrative Sources for Statistical Purposes Frameworks for the Access and Use of Administrative Data](https://reader031.vdocuments.mx/reader031/viewer/2022032705/56649da25503460f94a8e4b7/html5/thumbnails/24.jpg)
Formal Agreements
• May not be legally enforceable, but are seen as a formal commitment
• Set out terms for the provision and use of data
• Signed by all parties
• Can cover one or more data flows (e.g. Government contracts)
![Page 25: The Use of Administrative Sources for Statistical Purposes Frameworks for the Access and Use of Administrative Data](https://reader031.vdocuments.mx/reader031/viewer/2022032705/56649da25503460f94a8e4b7/html5/thumbnails/25.jpg)
Informal Agreements
• In a recent inventory, the British statistical system found it used over 1600 administrative data sets
• Many agreements were informal
• This increases the risk of problems, particularly if staff in either organisation move
![Page 26: The Use of Administrative Sources for Statistical Purposes Frameworks for the Access and Use of Administrative Data](https://reader031.vdocuments.mx/reader031/viewer/2022032705/56649da25503460f94a8e4b7/html5/thumbnails/26.jpg)
Group Exercise
What information should be included in a formal agreement?
![Page 27: The Use of Administrative Sources for Statistical Purposes Frameworks for the Access and Use of Administrative Data](https://reader031.vdocuments.mx/reader031/viewer/2022032705/56649da25503460f94a8e4b7/html5/thumbnails/27.jpg)
Agreement Contents 1
• Legal basis
• Names of persons transferring and receiving data
• Detailed description of data covered
• Frequency of data supply
• Quality standards
• Confidentiality rules
![Page 28: The Use of Administrative Sources for Statistical Purposes Frameworks for the Access and Use of Administrative Data](https://reader031.vdocuments.mx/reader031/viewer/2022032705/56649da25503460f94a8e4b7/html5/thumbnails/28.jpg)
Agreement Contents 2
• Technical standards
• Provision of metadata
• Provisions for payment for supply data
• Period of agreement
• Contingencies for changes in circumstances
• Procedure for resolving disputes
![Page 29: The Use of Administrative Sources for Statistical Purposes Frameworks for the Access and Use of Administrative Data](https://reader031.vdocuments.mx/reader031/viewer/2022032705/56649da25503460f94a8e4b7/html5/thumbnails/29.jpg)
Potential Challenges
• Financial agreements need to be negotiated
• Statistical institute may lack funds for data
• Suppliers may see data as having a commercial value
• Suppliers see no benefit to them
![Page 30: The Use of Administrative Sources for Statistical Purposes Frameworks for the Access and Use of Administrative Data](https://reader031.vdocuments.mx/reader031/viewer/2022032705/56649da25503460f94a8e4b7/html5/thumbnails/30.jpg)
Possible Solutions
• Statistical law to give the statistical institute the right to obtain data
• Statisticians could offer expertise, analysis of data, or the use of statistical tools in exchange
• Coordination with other customers in the statistical institute or government could help to spread the cost
![Page 31: The Use of Administrative Sources for Statistical Purposes Frameworks for the Access and Use of Administrative Data](https://reader031.vdocuments.mx/reader031/viewer/2022032705/56649da25503460f94a8e4b7/html5/thumbnails/31.jpg)
Quality Issues
• Data need to be ‘fit for purpose’
• Priorities of administrative sources and statistical offices differ
• Administrative bodies may spend little or no time on validation work, particularly on data items that are not key to their purpose
![Page 32: The Use of Administrative Sources for Statistical Purposes Frameworks for the Access and Use of Administrative Data](https://reader031.vdocuments.mx/reader031/viewer/2022032705/56649da25503460f94a8e4b7/html5/thumbnails/32.jpg)
Quality Conditions
• Have to be acceptable to both sides
• Could include:– Content of records (e.g. number of
items incorrectly classified should be less than 2%)
– Frequency and timeliness
– Provision of metadata
• Machinery for dealing with disputes
![Page 33: The Use of Administrative Sources for Statistical Purposes Frameworks for the Access and Use of Administrative Data](https://reader031.vdocuments.mx/reader031/viewer/2022032705/56649da25503460f94a8e4b7/html5/thumbnails/33.jpg)
TechnicalFrameworks
![Page 34: The Use of Administrative Sources for Statistical Purposes Frameworks for the Access and Use of Administrative Data](https://reader031.vdocuments.mx/reader031/viewer/2022032705/56649da25503460f94a8e4b7/html5/thumbnails/34.jpg)
Technical Frameworks• Mechanisms for data transfer
– Paper– Magnetic media– Secure on-line connection
• File formats
• Data / metadata standards– XML– SDMX– GESMES
![Page 35: The Use of Administrative Sources for Statistical Purposes Frameworks for the Access and Use of Administrative Data](https://reader031.vdocuments.mx/reader031/viewer/2022032705/56649da25503460f94a8e4b7/html5/thumbnails/35.jpg)
Case Study
Administrative Data for the British Statistical
Business Register
![Page 36: The Use of Administrative Sources for Statistical Purposes Frameworks for the Access and Use of Administrative Data](https://reader031.vdocuments.mx/reader031/viewer/2022032705/56649da25503460f94a8e4b7/html5/thumbnails/36.jpg)
Legal Framework
• No Statistics Act
• Access to administrative data controlled by source-specific acts
• Subject to European Union legislation
• Legislation for the independence of the statistical office may increase access
![Page 37: The Use of Administrative Sources for Statistical Purposes Frameworks for the Access and Use of Administrative Data](https://reader031.vdocuments.mx/reader031/viewer/2022032705/56649da25503460f94a8e4b7/html5/thumbnails/37.jpg)
Policy Framework
• Government policy favours more data sharing
• Requirement to cut costs and the burden on businesses
• Strong data protection legislation• Civil liberties groups suspicious of
government data sharing schemes
![Page 38: The Use of Administrative Sources for Statistical Purposes Frameworks for the Access and Use of Administrative Data](https://reader031.vdocuments.mx/reader031/viewer/2022032705/56649da25503460f94a8e4b7/html5/thumbnails/38.jpg)
Policy Framework
• Codes of Practice– Britain applies international standards
– National Code of Practice
– Supported by protocols, e.g. Protocol on Managing Respondent Load
• Statistical business register is the gateway for all administrative micro-data on businesses
![Page 39: The Use of Administrative Sources for Statistical Purposes Frameworks for the Access and Use of Administrative Data](https://reader031.vdocuments.mx/reader031/viewer/2022032705/56649da25503460f94a8e4b7/html5/thumbnails/39.jpg)
Organisational Framework
• Service Level Agreements with other government departments– reviewed annually– signed at director level
• Statistical analyses / tools provided in return (e.g. automatic coding software)
• Contracts with commercial data suppliers
![Page 40: The Use of Administrative Sources for Statistical Purposes Frameworks for the Access and Use of Administrative Data](https://reader031.vdocuments.mx/reader031/viewer/2022032705/56649da25503460f94a8e4b7/html5/thumbnails/40.jpg)
Organisational Framework
• Data suppliers represented on business register management committee
• Improving working relationships across government - ‘Business Data Sharing Group’
![Page 41: The Use of Administrative Sources for Statistical Purposes Frameworks for the Access and Use of Administrative Data](https://reader031.vdocuments.mx/reader031/viewer/2022032705/56649da25503460f94a8e4b7/html5/thumbnails/41.jpg)
Technical Framework
• Mostly quarterly copies of administrative registers on CD-ROM
• Government Secure Intranet link for daily updates from Value Added Tax
• Flat text files
• Metadata as separate tables
![Page 42: The Use of Administrative Sources for Statistical Purposes Frameworks for the Access and Use of Administrative Data](https://reader031.vdocuments.mx/reader031/viewer/2022032705/56649da25503460f94a8e4b7/html5/thumbnails/42.jpg)
Data Sources
Four main administrative sources:
• Value Added Tax (VAT) data• PAYE (income tax) data• Company registration data• Company ownership data (from a private sector source - Dun & Bradstreet)
![Page 43: The Use of Administrative Sources for Statistical Purposes Frameworks for the Access and Use of Administrative Data](https://reader031.vdocuments.mx/reader031/viewer/2022032705/56649da25503460f94a8e4b7/html5/thumbnails/43.jpg)
• Framework: VAT Act / Formal agreement
• Frequency: Daily, weekly, monthly
• Timeliness: Delivery within 5 days
• Quality: Good
• Delivery: Electronic via secure intranet
• Key content: Name, address, NACE, turnover
Value Added Tax Data
![Page 44: The Use of Administrative Sources for Statistical Purposes Frameworks for the Access and Use of Administrative Data](https://reader031.vdocuments.mx/reader031/viewer/2022032705/56649da25503460f94a8e4b7/html5/thumbnails/44.jpg)
Income Tax (PAYE) Data
• Framework: Finance Act / Formal agreement
• Frequency: Quarterly
• Timeliness: Within 1-4 months
• Quality: Employment - good
Address - moderate
Classification - poor
• Delivery: CD-ROM
• Key content: Address, NACE, employment
![Page 45: The Use of Administrative Sources for Statistical Purposes Frameworks for the Access and Use of Administrative Data](https://reader031.vdocuments.mx/reader031/viewer/2022032705/56649da25503460f94a8e4b7/html5/thumbnails/45.jpg)
Company Registration Data
• Framework: Contract• Frequency: Quarterly updates, continuous
on-line access• Timeliness: Good• Quality: Good• Delivery: CD-ROM / Internet • Key content: Legal name, company number
![Page 46: The Use of Administrative Sources for Statistical Purposes Frameworks for the Access and Use of Administrative Data](https://reader031.vdocuments.mx/reader031/viewer/2022032705/56649da25503460f94a8e4b7/html5/thumbnails/46.jpg)
Dun & Bradstreet Data• Framework: Contract
• Frequency: Annual update, quarterlysupplement, continuous
on-line access
• Timeliness: Good
• Quality: Variable
• Delivery: CD-ROM / Internet
• Key content: Enterprise group links, foreign ownership codes
![Page 47: The Use of Administrative Sources for Statistical Purposes Frameworks for the Access and Use of Administrative Data](https://reader031.vdocuments.mx/reader031/viewer/2022032705/56649da25503460f94a8e4b7/html5/thumbnails/47.jpg)
Group Discussion
What national frameworksdo you have in place?
How effective are they?