statistik.at Seite 1

Norbert Rainer

Quality Reporting and Quality Indicators

for Statistical Business Registers

European Conference on Quality in Official Statistics Rome, 8 - 11 July 2008

Special topic session 21

statistik.at Seite 2

Background and motivation

Character of a statistical business register

Some criteria for quality indicators

Example Accuracy

Concluding remarks

Overview

statistik.at Seite 3

Article 6 of the new BR – Regulation ( OJ L61, 5.3.2008, p. 6-16): 

Quality standards and reports

(1) MS take all measures to ensure the quality of the BR

(2) MS provide to the Commission (Eurostat) with a report on the quality of the BR

(3) Details of the quality report to be adopted in accordance with the regulatory procedure with scrutiny

(4) MS inform the Commission (Eurostat) on major changes affecting the quality of the BR

Background and motivation (1)

statistik.at Seite 4

Eurostat Business register unit:

-BR annual inquiry: included earlier few pilot questions on quality

-BR Recommendations Manual: Chapter 10 Quality Policy

• describes the quality dimensions (one additional: completeness)

• explains causes of quality defects

• reviews instruments of quality measurement

• outlines quality improvement strategies

• gives examples of quality indicators

Background and motivation (2)

statistik.at Seite 5

Eurostat: development of standard quality indicators

-The standard quality indicators:

Background and motivation (3)

Quality component

Indicator

Relevance R1. User satisfaction index R2. Rate of available statistics

Accuracy

A1. Coefficient of variation A2. Unit response rate (un-weighted/weighted) A3. Item response rate (un-weighted/weighted) A4. Imputation rate and ratio A5. Over-coverage and misclassification rates A6. Geographical under-coverage ratio A7. Average size of revisions

statistik.at Seite 6

Background and motivation (4)

Timeliness and Punctuality

T1. Punctuality of time schedule of effective publication T2. Time lag between the end of reference period and the date of first results T3. Time lag between the end of reference period and the date of the final results

Accessibility and clarity

AC1. Number of publications disseminated and/ or sold AC2. Number of accesses to databases AC3. Rate of completeness of metadata information for released statistics

Comparability

C1. Length of comparable time-series C2. Number of comparable time-series C3. Rate of differences in concepts and measurement from European norms C4. Asymmetries for statistics mirror flows

Coherence CH1. Rate of statistics that satisfies the requirements for the main secondary use

Source: Doc.ESTAT/02/Quality/2005/9/Quality indicators

statistik.at Seite 7

Background and motivation (5)

- The ESS Guidelines for quality reports (draft version 2)

With respect to quality indicators a distinction is made between the following six categories of statistics:

• Sample surveys

• Censuses

• Statistics derived from administrative sources

• Surveys involving data from multiple sources

• Price indices

• Statistical compilationsSource: Doc.ESTAT/DDG-02/Quality/2008/05aa

statistik.at Seite 8

Character of a statistical BR (1)

Sample surveys

Censuses

Statistics derived from administrative sources

Surveys involving data from multiple sources

Price indices

Statistical compilations

not a sample at all and no survey

intention of full coverage, but not a survey

administrative sources are used, but a database

multiple sources may be used, but a database

not any similarities

not any similarities

Categories of statistics Statistical Business register

Statistical registers

statistik.at Seite 9

BR is an instrument for conducting surveys

BR provides links to administrative sources

BR is a statistical database of longitudinal and cross- sectional character

BR is a statistical product to which the quality dimensions

• Relevance

• Accuracy

• Timeliness and punctuality

• Accessibility and clarity

• Comparability and coherence

do also apply

Character of a statistical BR (2)

statistik.at Seite 10

Quality indicators should be based on straightforward

concepts so that international comparability could be achieved

Quality indicators should primarily refer to the quality of the statistical BR, not to the quality of the applied administrative data / register

Quality indicators should be selected and designed with a view to the main uses of the statistical BR

Quality indicators should not be restricted to certain quality measurement instruments

Some criteria for quality indicators (1)

statistik.at Seite 11

A set of few, but significant indicators seems more appropriate than a long list of indicators

However, certain differentiations (especially with a view to accuracy or timeliness) need to be made in order to take into account the “weight” for the overall quality assessment, e.g.:

•kind of units

•size classes

•whether sample unit or not

•level of classification

Some criteria for quality indicators (2)

statistik.at Seite 12

Example: Accuracy

Coverage Measures of under / over coverageMeasures of falsely active unitsMeasures of duplicates

Completeness Measures of missing values of key variables

Deliverability Measures of quality of address data

Sampling Measures of quality of activity codingMeasures of quality of other classification codingMeasures of quality of size class coding

Other processing aspects of Measures of wrong links to administrative data

Measures of wrong unit structures etc.

statistik.at Seite 13

Statistical registers are statistical instruments but also a statistics category of its own

Quality reporting guidelines should take this into account explicitly

Many countries have already produced quality indicators for their BR

Concluding remarks

statistik.at Seite 14

THE END

Thank you for your attention !