topic (iii): macro editing methods paula mason and maria garcia (usa) unece work session on...
TRANSCRIPT
Topic (iii): Macro Editing Methods
Paula Mason and Maria Garcia (USA)
UNECE Work Session on Statistical Data EditingLjubljana, Slovenia, 9-11 May 2011
Topic (iii): Introduction
This topic covers issues concerning macro editing and selective editingMacro editing
Key Invited paper – AustraliaInvited papers – Netherlands, New Zealand,
Canada (2)Selective editing
Key Invited paper – SpainInvited papers – Sweden, UK
2
3
Topic (iii): Introduction
Macro-editing – WP.13 – significance editing framework for macro
editing – WP.14 – development of a macro editing tool– WP.15, WP.16, WP.17 – macro editing in an overall
editing strategy Selective editing– WP.18 – theoretical framework for selective editing– WP.19, WP.20 – selective editing using software tools
developed in Sweden, and applied by Sweden and the United Kingdom
Topic (iii): Macro Editing Methods
Enjoy the presentations!
4
Topic (iii): Macro Editing Methods
Summary of main developments and points for discussion
5
Macro editing: Main developments
WP.13 (Australia)– Added macro editing strategies to existing
significance editing framework – Scores based on predicting impact on outputs– Target macro editing effort at different
hierarchical levels – Incorporate sensitivity measures to address
swamping and masking
6
Macro editing: Main developments
WP.14 (Netherlands)– Software for developing custom macro editing
tools accessed by scripts– Functionalities include aggregation techniques,
data visualization, dynamic filters, data correction and recalculation.
7
Macro editing: Main developments
WP.15 (New Zealand) – Incorporate macro editing in an overall editing
strategy– Increased use of automatic micro edits– Prioritize using expected effects on the outputs– Developed quality indicators– Report efficiency gains
8
Macro editing: Main developments
Canada – Common survey framework for business surveys (two papers)WP.16– Iterative process – Rolling estimates model and common editing strategy– Elimination of manual intervention until after estimates
are available– Allocation of resources based on macro quality
indicators and micro level scores
9
Macro editing: Main developments
(Continued)WP.17 – Shared, generic corporate strategies, methodologies, and
common metadata framework– Methodology for top down approach– Methodology for measuring quality and measures for
quality– Score functions to measure impact
10
Selective editing: Main developments
WP. 18 (Spain) – Theoretical framework for selective editing as an
optimization problem– Minimize expected workload subject to minimal
expected error on the aggregates– Linear constraints – computationally easier, suitable
when timeliness is an issue– Quadratic constraints – wider error bounds, more
units are marked for review
11
Selective editing: Main developments
SELEKT tools at both Statistics Sweden and ONS– Scores based on suspicion, potential impact on the
outputs– Need “expected” values, final data from previous cycle
WP.19 (Sweden) – Prioritize using expected effects on the outputs– “Expected “ values using time series or cross-sectional
data – Different levels of data edited concurrently
12
Selective editing: Main developments
WP.20 (UK) – Selective editing as part of an overall efficient editing
strategy– Assess impact on quality of changes to edit rules prior
to using SELEKT– Suspicion based on traditional edit rules or test
variables
13
Points for discussion
Using a software tool and/or scores for guiding macro editing operations and/or selective editing has benefits: standardizes review process, can be used for several surveys, and provides overall cost benefits. – How are agencies incorporating cost/resources savings into the
survey process? – How are agencies planning on maintaining these tools/systems
given the complexities of the metadata, constraints, variable mappings, expectation models, and hierarchies as surveys and output requirements evolve (particularly business surveys)?
14
Points for discussion
(Continued)– What is the effect on other survey activities? – How is the overall macro editing and/or
selective editing process contributing to the overall data quality?
– How can the effect on data quality be measured?
15
Points for discussion
When macro editing and/or selective editing tools are applied to periodic survey data, subject matter experts may acquire further knowledge about the survey from the macro editing and/or selective editing operations: – How can this knowledge be used to improve the survey
process? – How can we incorporate this knowledge to get insight into
how to reduce errors and/or enhance micro editing for the next cycle?
16
Points for discussion
In both macro editing and selective editing scores there is the need for estimates of anticipated values. – How to model “expected” values needed for computing
measures of suspicion and/or impact? – How do we choose the appropriate domains for
computation of “expected” values in order to achieve relevancy and accuracy?
– What is the minimum number of observations needed to compute these “expected” values within each domain?
17
Points for discussion
(Continued)– How do we separate model errors for expected values
from response errors (for either aggregate expected values or micro expected values) in a production environment?
– Are there concerns about potential bias under certain variable distributions that may result from a collection of non-influential units that will not be addressed by selective editing?
18
Point for discussion
Most statistics may benefit from the use of macro editing and/or selective editing. – What are the agencies specifications for a set of
general mandatory guidelines?
19
Point for discussion
When designing an overall editing strategy, – To what extent should agencies incorporate selective
editing and/or macro-editing in their overall editing strategies?
– For what kind of data are these strategies suitable?– How can we take into account the fact final data may be
used by other users and for different purposes?
20
21
Thank you for your attention!
Paula and Maria