© federal statistical office germany, division ib, institute for research and development in...

Sheet 1© Federal Statistical Office Germany, Division IB, Institute for Research and Development in Federal Statistics

Surveys, administrative data or integrated

models: A decision by quality indicators?

Jörg Enderer, Dieter Schäfer

European Conference on Quality in Official Statistics

Q2010, Helsinki, 4-6 May 2010


The project: Use of administrative data in short-term statistics

Turnover tax data from fiscal authorities

Employment data from Federal

Employment Agency

monthly data:short term statistics

other administrative

data

annually data:business register

data from statistical surveys


Reshaping STS - Results2004

S

U

R

V

E

Y

S

Full replacement of surveys: Craft statistics (German particularity)

Integrated model of smaller survey and administrative data: Service sector, wholesale trade, maintenance of motorvehicles

Surveys unchanged, additional use for non-covered enterprises: Building installation

Surveys unchanged: Manufacturing, site preparation and civil engineering, retail trade, hotels and restaurants

2011


Favorable testing conditions

Availability of aggregated results and micro data for surveys and administrative data in terms of

periods economic activities regions

Extensive possibilities to compare the different methods for the same field

survey administrative data integrated models of survey and administrative data


Questions (ex post)

Which quality criteria were relevant for the choice of the

method?

Could the criteria be described by quality indicators? By

indicators included in ESQR?

Did the indicators lead to a clear decision for a certain

method?


Relevant quality criteria

Relevance

Accuracy

Timeliness

Punctuality

Comparability

Accessibility

Clarity

(Coherence)

Important criteria for the tests

Criteria independent from choice of method


Quality criterion: Relevance

Target: User needs have to be fulfilled Surveys: Do we get what we want?

Check of microdata => Accuracy issue Administrative data:

Differences in definitions, statistical units, allocation to economic activities

Quantitative indicators: Differences in concepts in % No uniform results for different activities and regions Elimination of important differences by (complex) estimation models

=> Problems of accuracy?

Role of indicators: Information on big deviations of administrative data; no quantitative indicators for smaller deviations in definitions


Quality criteria: Timeliness and punctuality

Target: Comply with existing standards of timeliness

Indicators: Time lag in days; additional indicators for subprocesses

How does the fulfillment of the target affect the accuracy?

Administrative data:

Suitable and automatic IT procedures had to be developed

Higher time restrictions for editing and estimation methods

Administrative Data: In some cases target not achievable with reasonable accuracy, e.g. t+30 for manufacturing, retail trade

Role of Indicators: Minimum Standards


Quality criterion: Accuracy Targets: Assure accuracy of the status quo

Different dimensions of accuracy:

Internal comparision: Revisions for each method

Comparison of results of different methods

Accuracy of estimations for administrative data

ESQR - Indicators: Average size of revisions, response rates, edit failure rates, coefficient of variation, coverage rates

Additional indicators for administrative data like

misclassification rates, linkage rates, risk indicators for differences in statistical units (tax groups)


Accuracy: Limits of indicators

Size of revisions Surveys: Correction of missing data and outliers Administrative data: Correction of missing data, outliers and

update of data

Higher size of revisions does not necessarily mean poorer quality

Does higher edit failure rate mean less accuracy?

Different indicators are difficult to weight against each other: Coefficient of variation against misclassification?

Role of indicators: Vital but not without sound interpretation


Decision-making by indicators? Minimum standards for single quality criteria are important

Minimum standards for relevance and timeliness can be reached at the expense of accuracy

Accuracy is very complex and difficult to judge for one method

(esp. non sampling errors)

Comparison of accuracy for different methods even more difficult

ESQR – Indicators are very important, but administrative data need partly other (additional) indicators

Quantitative indicators need background information for sound interpretation

Composite/overall quality indicators are no way out


Decision-making process in the project

1. The decision was a social process managed by written summaries, discussions between producers and communication with main users

2. Standards on relevance, timeliness and punctuality must be met

3. Minimum standards for accuracy must be ensured, e.g. acceptable

size of revisionsestimation ratescoefficients of variation

4. In addition pros and cons have to be balanced.


Dieter Schäfer and Jörg Enderer

E-Mail: [email protected]

Many thanks for your attention

Any questions?

© federal statistical office germany, division ib, institute for research and development in...

Documents