evaluating the effects of business register updates on monthly survey estimates daniel lewis
TRANSCRIPT
![Page 1: Evaluating the Effects of Business Register Updates on Monthly Survey Estimates Daniel Lewis](https://reader035.vdocuments.mx/reader035/viewer/2022062618/5514bd07550346b0338b46c1/html5/thumbnails/1.jpg)
Evaluating the Effects of Business Register Updates on Monthly
Survey Estimates
Daniel Lewis
![Page 2: Evaluating the Effects of Business Register Updates on Monthly Survey Estimates Daniel Lewis](https://reader035.vdocuments.mx/reader035/viewer/2022062618/5514bd07550346b0338b46c1/html5/thumbnails/2.jpg)
Overview
• Introduction
• Different strategies for updating the business register
• Simulation method for testing updating strategies
• Simulation results
• Conclusions and comments
![Page 3: Evaluating the Effects of Business Register Updates on Monthly Survey Estimates Daniel Lewis](https://reader035.vdocuments.mx/reader035/viewer/2022062618/5514bd07550346b0338b46c1/html5/thumbnails/3.jpg)
Introduction
• Inter-Departmental Business Register (IDBR) used by most
ONS business surveys
• Key variables – employment, turnover, industry (SIC)
• Updated from different survey and admin sources
• Quality of updates affects survey estimates
• Desire to produce priority rules for updating
• Accuracy of updating strategies tested by simulation
![Page 4: Evaluating the Effects of Business Register Updates on Monthly Survey Estimates Daniel Lewis](https://reader035.vdocuments.mx/reader035/viewer/2022062618/5514bd07550346b0338b46c1/html5/thumbnails/4.jpg)
Sources for updating the IDBR
• Employment:– Pay As You Earn tax data (PAYE)– Business Register Survey (BRS)– Imputed from turnover data
• (Annual) Turnover:– Value Added Tax data (VAT)– Imputed from employment data
• Standard Industrial Classification (SIC):– VAT– PAYE– BRS
![Page 5: Evaluating the Effects of Business Register Updates on Monthly Survey Estimates Daniel Lewis](https://reader035.vdocuments.mx/reader035/viewer/2022062618/5514bd07550346b0338b46c1/html5/thumbnails/5.jpg)
Updating scenarios tested
• Employment:– Always favour PAYE– Always favour BRS– Favour BRS if less than 3 / 2 / 1 years old
• SIC:– Always favour PAYE– Always favour VAT– Always favour BRS– Favour BRS if less than 3 / 2 / 1 years old– Range of options for second priority if first unavailable
• Frequency of updates – monthly, quarterly, annually
![Page 6: Evaluating the Effects of Business Register Updates on Monthly Survey Estimates Daniel Lewis](https://reader035.vdocuments.mx/reader035/viewer/2022062618/5514bd07550346b0338b46c1/html5/thumbnails/6.jpg)
Method for testing updating scenarios
• Assess effect on monthly turnover survey estimates
• Simulation method using four steps:
1. Simulate 12 months ‘real world’ data
2. Create business register
3. Select and survey samples from register
4. Calculate estimates and compare to ‘true’ values
![Page 7: Evaluating the Effects of Business Register Updates on Monthly Survey Estimates Daniel Lewis](https://reader035.vdocuments.mx/reader035/viewer/2022062618/5514bd07550346b0338b46c1/html5/thumbnails/7.jpg)
1. Simulate ‘real world’ data (i)
• Use January IDBR data as starting point
• Analyse 12 months of IDBR data to estimate:– Probability of a business dying– Probability of changing SIC– Probability of a change in turnover– Probability of a change in employment– Probability of a new business being born
• Probabilities used to randomly assign characteristics to businesses each month
![Page 8: Evaluating the Effects of Business Register Updates on Monthly Survey Estimates Daniel Lewis](https://reader035.vdocuments.mx/reader035/viewer/2022062618/5514bd07550346b0338b46c1/html5/thumbnails/8.jpg)
1. Simulate ‘real world’ data (ii)
• Changes in employment and turnover modelled based on observed means and standard deviations within strata
• Change in SIC randomly assigned based on probabilities of each type of SIC change
• Births and deaths also randomly introduced based on probabilities
• Monthly turnover data created by comparison with weighted monthly survey data from the same year
monthly survey turnover estimatesimulated annual turnover
actual IDBR annual turnover
![Page 9: Evaluating the Effects of Business Register Updates on Monthly Survey Estimates Daniel Lewis](https://reader035.vdocuments.mx/reader035/viewer/2022062618/5514bd07550346b0338b46c1/html5/thumbnails/9.jpg)
2. Create business register
• Create register variables from ‘real world’ for each updating source with matching (realistic) quality deficiencies
• Quality parameters for each source derived by comparing average changes in employment and SIC before and after main register update
• Variables for each updating scenario derived by adding random variation to ‘real world’ value:
,
,
, ,
where unit value for a given source and age
real world value for unit
quality parameter for the source and age
N(0,1)
i sa
i real
sa
i
i sa sa ii real
Y i
Y i
Q
Z
Y Y Q Z
![Page 10: Evaluating the Effects of Business Register Updates on Monthly Survey Estimates Daniel Lewis](https://reader035.vdocuments.mx/reader035/viewer/2022062618/5514bd07550346b0338b46c1/html5/thumbnails/10.jpg)
3. Select and survey samples
• Samples drawn many times from the simulated business registers using typical business survey sample design
• Stratified by industry and employment
• Neyman allocation using annual turnover data
• Samples selected using Permanent Random Number sampling with different random start for each iteration and 15 month rotation period
• Turnover value ‘collected’ for each sampled business
![Page 11: Evaluating the Effects of Business Register Updates on Monthly Survey Estimates Daniel Lewis](https://reader035.vdocuments.mx/reader035/viewer/2022062618/5514bd07550346b0338b46c1/html5/thumbnails/11.jpg)
4. Calculate and compare estimates
• Estimates of total turnover calculated for each sample using separate ratio estimator with employment as auxiliary
• MSEs calculated for each updating scenario by comparison with true turnover in the ‘real world’:
2
1
ˆ( )N
ii
Y YMSE
N
![Page 12: Evaluating the Effects of Business Register Updates on Monthly Survey Estimates Daniel Lewis](https://reader035.vdocuments.mx/reader035/viewer/2022062618/5514bd07550346b0338b46c1/html5/thumbnails/12.jpg)
Simulation results
• Simulation very time consuming
– Only possible to draw 516 samples
– Just sufficient for convergence
• Best updating strategy:
– Update IDBR monthly
– Give preference to PAYE for employment
– Use BRS for SIC if less than 3 years old, otherwise VAT
• A few other options were not significantly worse
![Page 13: Evaluating the Effects of Business Register Updates on Monthly Survey Estimates Daniel Lewis](https://reader035.vdocuments.mx/reader035/viewer/2022062618/5514bd07550346b0338b46c1/html5/thumbnails/13.jpg)
Conclusions and comments
• Method very computer intensive, but gave useful results
• Project time limited, so constrained to using simple methods
• Potential to extend the model to better reflect business survey populations and updating processes
• Then possible to test a wide range of business survey methods:
– Sample designs, rotation rates, estimation, variance estimation, outlier treatment, …