data modification, data suppression, small populations and other features of the 1991 small area...

Data Modification, Data Suppression, Small Populations and Other Features of the 1991 SmallArea StatisticsAuthor(s): Keith ColeSource: Area, Vol. 26, No. 1 (Mar., 1994), pp. 69-78Published by: The Royal Geographical Society (with the Institute of British Geographers)Stable URL: http://www.jstor.org/stable/20003373 .

Accessed: 10/06/2014 19:50

Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at .http://www.jstor.org/page/info/about/policies/terms.jsp

.JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range ofcontent in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new formsof scholarship. For more information about JSTOR, please contact [email protected].

.

The Royal Geographical Society (with the Institute of British Geographers) is collaborating with JSTOR todigitize, preserve and extend access to Area.

http://www.jstor.org

This content downloaded from 188.72.127.85 on Tue, 10 Jun 2014 19:50:53 PMAll use subject to JSTOR Terms and Conditions

http://www.jstor.org/action/showPublisher?publisherCode=rgs

http://www.jstor.org/stable/20003373?origin=JSTOR-pdf

http://www.jstor.org/page/info/about/policies/terms.jsp


Area (1994) 26.1, 69-78

Data modification, data suppression, small populations and other features of the 1991 Small Area Statistics

Keith Cole, Census Dissemination Unit, Manchester Computing Centre, University of Manchester, Oxford Road, Manchester M13 9PL

Summary This paper describes the effect that data modification, data suppression, small populations, imputation and sampling error may have on the reliability of results based on an analysis of the 1991 Census Small Area Statistics at Enumeration District/Output Area level.

The 1991 Local Base Statistics (LBS) and Small Area Statistics (SAS) are a predefined set of cross-tabulations of two or more census variables which are made available by the Census Offices for a wide variety of different areal units throughout

Great Britain, including areas smaller than those reported in the published volumes (Cole 1993). The availability of the 1991 SAS down to Enumeration District (ED) (England and Wales) and Output Area (OA) (Scotland) level in a digital format will inevitably mean that a large number of users will be undertaking computer based analysis and mapping of 1991 Census statistics at this area level. However, there are a number of statistical and methodological problems associated with attempting to analyse and map SAS at such a small spatial scale. The five main problems associated with the Enumeration District and Output Area (ED/OA) level SAS which will be described in this article are related to data suppression, data modification, variations in population size, imputation and sampling error. It is important all census users should be aware of the potential impact of these problems as they may in some instances severely affect the reliability of any results based on an analysis of the SAS at ED/OA level.

Data suppression

For areas which may have very small populations, such as EDs/OAs, there is a small risk that some of the very detailed tables in the SAS might inadvertently reveal information about identifiable households or individuals. As with the 1971 and 1981 SAS, the two procedures adopted by the Census Offices to lessen the risk of inadvertent disclosure presented by areas with small populations are the modification of the 100 per cent statistics and the suppression of statistics for those areas falling below particular population thresholds.

In England and Wales, the 1991 SAS is only released for those EDs with 50 or more usually resident persons and 16 or more resident households. The SAS for those EDs failing to pass the minimum population thresholds will not be released

with the exception of three basic counts (total persons present, total residents and total resident households). In Scotland, the SAS have been released for all OAs since all provisional OAs which did not pass the minimum population thresholds were amalgamated with a predetermined contiguous area in the final stage of creating OAs.



70 Cole

For the 1981 SAS the minimum population thresholds were 25 usually resident persons and eight resident households. The person and household thresholds for the 1981 SAS were used separately which resulted in partial suppression of the SAS. For instance, if the number of usually resident persons was over 25 but the number of resident households was under eight, tables counting persons would be released but tables relating to households would be suppressed. The problems associated with the differential suppression of statistics are avoided with the 1991 SAS as the population thresholds, with the exception of Special EDs/OAs, are not applied separately.

The SAS for a suppressed 1991 ED in England and Wales are amalgamated with those of a contiguous ED, provided that the combined total numbers of persons and households exceeds the minimum population thresholds. The only exception to this rule is for those EDs with true zero populations which will not be subject to amalgamation. The SAS for the importing ED will always be for the 2 EDs combined. Therefore any user analysing the SAS will need to ascertain whether a zero population total in an ED is a true zero or the result of data suppression. Similarly, the user will also need to be aware of which EDs have imported SAS from one or more suppressed EDs.

For the 1981 SAS, only the 10 per cent statistics from suppressed EDs were amalgamated with those of an adjoining area. The failure to redistribute the 1981 100 per cent statistics from suppressed EDs resulted in the under-reporting of totals when one or more suppressed EDs were aggregated with other EDs to form larger output areas, such as wards and local authority districts.

Slightly different procedures have been adopted for dealing with 1991 Special EDs/OAs (SEDs/SOAs). SEDs and SOAs have been defined for those large communal establishments, such as prisons, educational establishments and hotels, which were expected to contain more than 100 or more persons on census night. Three basic counts (total persons present, total residents and total resident households) are released for all SEDs/SOAs. All the SAS are released for a SED/SOA with 50 or more residents and 16 or more resident households. In England and Wales, the SAS for a SED which has failed both population thresholds are not amalgamated with those of the containing ED but the populations are included in the SAS for ward level and above. In Scotland, a SOA failing both population thresholds has been merged with its surrounding OA rather than being suppressed. SAS tables counting residents are also issued for a SED/SOA with 50 or more residents but less than 16 resident households. As described above, the full or partial suppression of the SAS for SEDs/SOAs may result in an under-reporting of population totals when aggregating EDs to form larger output areas, such as wards of districts, which contain large numbers of whole or part suppressed SEDs/SOAs.

Data modification

In addition to suppression, a data modification technique is used to ensure that no information in the released LBS/SAS can be related to any identifiable household or

individual with any degree of certainty. Non-zero counts in the 100 per cent SAS at

ward/postcode sector level and below are modified by the addition of +1, 0 or - 1 in quasi-random patterns such that there is a higher probability of leaving the count unmodified. All counts are modified, with the exception of the basic population counts in tables 1, 27 and 71 and the counts of establishments in table 3 where



1991 census: small area statistics 71

modification would impair the usefulness of the tables. The counts in the 10 per cent

tables are not modified as sampling is regarded as providing sufficient protection against inadvertent disclosure.

Data modification can introduce two different types of perturbation into the SAS. First, each non zero cell will have been subject to the possible addition of 1, 0 or - 1. The effect of this data adjustment will be greatest when the value of the count is small which is also when the risk of disclosure would be greatest. For example, adding or subtracting 1 from a value of 5 has far greater proportionate effect than it does on values of 50 or 500. Therefore the effects of data adjustment on individual cell counts is most severe in areas with small populations and/or for those census variables with a low frequency of occurrence, and/or distributed over a large number of categories, such as single years of age.

Secondly, adding the modified cells together to form a table total or subtotal also has the effect of summing the individual random adjustments to produce a net error term. This net error term will be the difference between the modified and unmodified table totals or subtotals. For example, if a table total consists of five cell counts which have been modified through the addition of the following sequence of random errors (1, 0, - 1, - 1, - 1) then the net error term for that table total will be - 2. (It should be noted, that for the purposes of illustration, a more extreme sequence of data modification than normally occurs in practice has been used). Similarly, any user defined variables based on the aggregation of a series of individual counts will also be subject to net error.

In the 1981 and 1991 SAS, a small number of tables containing basic population counts, such as the total number of households and the total number of residents in each area, are not subject to any modification. By comparing these basic counts from

modified and unmodified tables it is possible to examine the extent to which net error can affect table totals.

Users may often fail to appreciate that net error terms may actually cumulate rather than automatically cancel out when EDs/OAs are aggregated. This can be demonstrated by using the 1991 ED level SAS for the Isle of Wight. For all EDs (excluding shipping EDs) in Medina (KY) and South Wight (KZ) local authority districts, the modified and unmodified total number of residents in households (counts S350001 and S010065, respectively) were aggregated to obtain ward level totals. The difference between these two totals represents the net error term. Individually, none of the EDs has an aggregate error term which exceeded ? 3. As

Table 1 illustrates, the net errors may actually cumulate when EDs are aggregated by users to obtain ward and/or district level statistics. For example, wards KYFA and KYFP have net errors of +23 and - 19 persons respectively. Similarly, the aggregate errors for districts KY and KZ and -6 and 40 persons respectively.

Caution is therefore required when aggregating ED/OA level SAS to obtain totals for larger output areas, such as wards/postcode sectors, districts, counties/regions or countries, since these totals will not necessarily be the same as those reported in the SAS or Census Reports for those area levels. For example, the total number of residents in households in the Isle of Wight (S350001) obtained by a user aggregating all the EDs in the county is 120,346 compared to 120,439 reported in the county level SAS and County Report. The difference between these two totals (-93 persons) is due to the combined effect of data adjustment (+34 persons) and the suppression of statistics for special EDs (- 127 persons).

While the differences between totals may appear small, their effect can be severe, particularly if rigid cut-off points are used for classifying areas. For example, as



72 Cole

Table 1 Effect of aggregation on the cumulative net error term for SAS table S35

Total residents Number in households Net

Ward ID of EDs S010065 S3500101 Error

Medina District: KYFA 9 3,519 3,542 23 KYFB 5 2,514 2,522 8 KYFC 7 3,157 3,154 -3 KYFD 9 3,678 3,676 - KYFE 7 3,831 3,831 0 KYFF 10 5,106 5,096 -10 KYFG 5 1,719 1,718 - 1 KYFH 7 3,106 3,099 -7 KYFJ 8 4,333 4,348 -15 KYFK 10 3,476 3,488 12 FYFL 14 6,102 6,099 -3 KYFM 6 2,917 2,915 -2 KYFN 14 5,624 5,626 2 KYFP 14 5,515 5,496 - 19 KYFQ 12 3,943 3,935 -8 KYFR 11 4,894 4,885 -9 KYFS 13 5,499 5,497 -2 KY 161 68,933 68,927 -6

South Wight District: KZFA 8 3,502 3,505 3 KZFB 11 3,383 3,396 13 KZFC 5 2,063 2,066 3 KZFD 7 2,411 2,414 3 KZFE 8 2,321 2,322 1 KZFF 7 2,747 2,747 0 KZFG 15 5,081 5,087 6 KZFH 6 2,539 2,538 - 1 KZFJ 10 4,341 4,350 9 KZFK 16 4,930 4,934 4 KZFL 11 3,872 3,871 - 1 KZFM 11 3,516 3,519 3 KZFN 6 2,431 2,431 0 KZFP 14 5,723 5,725 2 KZFQ 4 1,642 1,639 -3 KZFR 3 877 875 - 2

KZ 142 51,379 51,419 +40

Source: 1991 Census, ESRC Purchase, Crown Copy

described by Senior (1991), the ward level deprivation index used by the Department of Health was originally based on aggregated 1981 ED level SAS. When the deprivation index was recalculated using ' true' ward level SAS it was discovered that nineteen out of 8,465 wards in England had been misclassified which cost the

Department of Health an extra ?190,000 in deprivation payments in 1991.




As a guideline, the ED/OA level SAS should not be aggregated to obtain population totals for area levels which form part of the standard output, such as wards, districts, District Health Authorities or Parliamentary Constituencies where the user has access to such output. In addition, when aggregating the SAS to form user defined new zones which do not form part of the standard output, the highest possible area level(s) should be used as the basic building blocks for the aggregation.

Moreover, using 100 per cent SAS tables to calculate ratio measures requires even more caution as both the numerator and the denominator will have been subject to data adjustment, particularly if these are based on the aggregation of a number of counts. For areas with small populations the combined effect of data adjustment on individual counts and table totals can have a highly distorting effect on ratio

measures. For instance, the following counts have been extracted for ED ELFA05 in Cheshire:

HHLDS S420001 S420020 ELFA05 22 27 2

Number of households lacking or sharing use of bath/shower and/or inside S4220020 WC containing 1 adult of pensionable age. S420001 Total number of households with residents-modified total (total 42)

HHLDS Total number of households with residents-unmodified total (table 27)

First, as S420020 has been subject to data modification the unmodified count of the number of households lacking or sharing use of a bath/shower and/or inside WC and containing 1 adult of pensionable age must be either 1, 2 or 3. Using HHLDS as the denominator, the 'true' percentage of households lacking or sharing use of bath/shower and/or inside WC and containing one adult of pensionable age will be either 4 5 per cent, 9-1 percent or 13 6 percent. However, when S420020 is expressed as a ratio of the table total S420001, which is 5 greater than the unmodified total, a percentage figure of 7 4 per cent is produced. Conversely, had the table total been 5 less than the unmodified total a percentage figure of 11 7 per cent would have been obtained.

These wide variations (4 5 per cent-13 6 per cent) clearly illustrate the distorting effect that data modification can have on the calculation of ratios for areas with very small populations. While this may be an extreme example, it does illustrate how it can be very difficult to decide whether a high or low concentration is real or the result of a statistical accident. It also makes it extremely difficult to identify areas of disadvantage solely by looking at the extremes of distributions since the amount of error at the extremes of the distribution may be greater than in the middle.

A further complication arises as a result of the considerable data duplication that exists within the SAS which means that it is possible to calculate the same ratio

measures, such as the percentage of households in council tenure, using a number of different tables. As the combined effect of data modification on individual cell counts and the table totals may vary from table to table, it is possible that a wide range of different results may be obtained depending on which table is used. Unfortunately, it is not possible to measure the extent of the net error term for all tables, as only a small percentage of unmodified population bases are reported in the SAS. Similarly, due to the differential effect that data modification may have on different tables, users



74 Cole

should be wary of using numerators and denominators from different tables to avoid the potential problem of obtaining percentage values greater than 100 per cent.

Variations in population size

As measured by the number of residents or resident households there is considerable variation in the size of 1991 EDs/OAs. For the 1991 Census, Scotland differs from England and Wales in terms of the size of the smallest unit for which the SAS are released. Whereas the Census Offices reduced the number of smaller EDs for enumeration purposes, the number of small output areas in Scotland increased as

Output Areas became separate from enumeration areas (Clark and Thomas 1990). It is estimated that OAs in Scotland will have population sizes averaging only one third the size of EDs in England and Wales. Such a pronounced national variation may have a significant effect on any analysis of the ED/OA level SAS, such as an area classification or the identification of deprived areas, undertaken for Great Britain as a whole. OAs could be aggregated together to form larger, more comparably sized intermediate units, such as 1981 EDs, but, as described above, the resultant SAS

would contain much larger standard errors. In order to be able to compare areas in terms of their census characteristics, it may

be necessary to control for variations in population size. One standard method used to do this is to express a count as a percentage of a particular population base. Using such percentages to identify areas with high or low concentrations of particular characteristics, such as male unemployment, is a widely used technique. For example, the Department of Environment (1983) used Z-scores based on percentage data to identify deprived areas using 1981 SAS. As described above, data adjustment can severely affect ratio measures calculated for EDs/OAs areas with small populations. As a result, using percentages to identify areas with high or low concentrations can produce highly misleading results. For example, consider the following two hypothetical EDs (EDOI and ED02):

Total Council owned Total private households households

EDOI 8 16 ED02 100 200

In both EDs, the percentage of households in council ownership is identical (50 per cent) but in view of the greater absolute numbers involved we can be more certain that the concentration in ED02 is not the result of a statistical artifact, caused through the effects of data modification.

One possible solution to the problems posed by varying ED/OA size and data adjustment is to restrict analysis to those EDs/OAs above a certain population threshold, such as 50 households (Holterman 1975). However, this threshold would cause problems in Scotland where the average size of a 1991 Output Areas is

approximately 52 households. Another alternative approach is the use of the signed chi-square transformation

X2). This technique, which was first used for Census mapping in People in Britain

(CRU/OPCS/GRO(S) 1980), has been quite widely advocated (Visvalingham 1978;




Jones and Kirby 1980; Rhind 1983; Morphet 1992). The signed Z2 statistic, which is calculated using the formula

x2=_ (O-E)2

E

(where 0 and E are the observed and expected numbers respectively), can be used to identify areas which have high or low concentrations based on the expected and observed numbers within an area. The x2 statistic is given a positive or negative value depending on whether the observed value is greater or less than the expected number. A value of zero indicates that the observed and expected values are identical.

The main advantage of the signed x2 statistic over percentages is that it takes into account both relative and absolute values and as a result it can provide a useful technique for compensating for the distorting effect that data adjustment can have on the calculation of ratio measures for EDs with small populations. An ED with a small population must have extremely large concentrations in order to have a high x2 value.

Any ranking of EDs based on x2 statistic may be substantially different to that based on percentage figures. Therefore, the decision whether to use percentages or the signed x2 statistic could have an important effect on the outcome of exercises

which use ED/OA level SAS to identify where the most deprived EDs/OAs in Great Britain are located. Any ranking of EDs/OAs based on percentages may give greater weighting to those areas of Great Britain, such as Scotland, with higher proportions of small EDs/OAs. Conversely, a ranking based on the signed x2 statistic

may tend to put areas with larger populations at the extremes. Both the percentage and the signed x2 statistic are techniques that can be used to

identify those EDs/OAs with the highest concentrations of particular groups, such as unemployed males or lone parent families. However, they may not be effective in identifying where the majority of the members of a particular group live. For example, most of the unemployed males in Great Britain in 1981 (approximately 70 per cent) did not live in the 10 per cent of EDs with the highest levels of male unemployment. Therefore, for certain purposes, such as targeting resources to areas on a per capita basis, it may be more appropriate to identify the absolute numbers of particular individuals and/or households in an area. Similarly, by ranking EDs/OAs by absolute numbers it is possible to identify the extent to which particular groups, such as lone parent families or pensioner households lacking central heating, are spatially concentrated Uones and Moon 1987).

Imputation

For the first time in 1991, the Census Offices have used imputation procedures to obtain 100 per cent statistics for households which were considered to be usually resident but were not enumerated. At the time of writing, very little is known about the effect that variations in imputation rates has on the reliability of the ED/OA level SAS and, in particular, when making estimations of 100 per cent values from the 10 per cent sample statistics. Preliminary investigations reveal that in some local authority districts there are considerable variations between EDs/OAs in terms of the level of imputation. For example, within one ward in Manchester, the imputation rates by ED for residents in households vary from 1 per cent of total residents to 53 2 per cent. Although this may be an extreme example, it does illustrate how users need to be aware of variations in imputation rates when analysing ED/OA level SAS. In this context, tables 1, 19 and 71 in the SAS can be used to obtain the number



76 Cole

of imputed absent households and residents in each ED/OA. Further details about imputation procedures are provided in the Census Definitions volume (HMSO 1992).

Sampling error

Statistics in section VI of the LBS/SAS are based on a 10 per cent sample of households (excluding imputed households) and a 10 per cent sample of persons in communal households. Due to sampling error, statistics derived from the 10 per cent sample can only provide estimates of the 100 per cent figures. Although the geographically stratified sample of households will tend to reduce the degree of sampling error, this may be offset by the effect of clustering within households. It is known that persons with similar characteristics, such as ethnicity or educational level, tend to cluster together within households and as a result this may serve to increase the sampling error, particularly for groups of the population with rare characteristics.

An investigation into the sampling errors associated with the 1981 Census (OPCS 1983 and OPCS 1985) confirmed that the 1981 10 per cent sample statistics could be grossed up by a factor of 10 to provide reliable estimates of the 100 per cent population for large areas such as local authority districts. However, for areas with small populations such as wards and EDs the 10 per cent statistics were subject to large sampling errors and grossing up was not advisable at these area levels. The 1991 10 per cent sample statistics can be grossed up, but not by a simple factor of 10, since imputed households have been excluded from the sample. Using a grossing factor of 10 will only provide an estimate of the enumerated population. Due to the effect of imputation, the grossing factor for the 10 per cent sample will be over 10 and may also vary from area to area.

The existence of potentially large sampling errors in the ED/OA level 10 per cent SAS means that unreliable results may be produced when the data are analysed or

mapped at such a small spatial scale. Therefore the ED/OA level 10 per cent SAS must always be aggregated to form much larger areas in order to produce more reliable estimates of the 100 per cent population. For example, using 1981 10 per cent SAS it was necessary to aggregate together all the 363 EDs in Macclesfield

District to produce a sample value which was within 5 per cent of its true value (68 per cent confidence level) for the number of households where the economically active head of the household was in social class I. The 10 per cent SAS have only been released at ED/OA level to provide users with a primary building block which can be used as a basis forflexible area aggregations.

Users should also be aware of how sample errors can affect the reliability of percentage figures obtained using the 10 per cent sample statistics. The following formula (Weis and Hassett 1991; Butcher and Elliot 1986) can be used to calculate the standard error (SE) of a percentage obtained from sample data

SE(p) / n n

where p is the percentage and n is the sample size. The important point to note from this formula is that the standard error is based on the value of the percentage as well as the sample size. Thus for a given sample size, the standard error of a percentage is greatest when it has a value of 50 per cent and lowest for values of 0 per cent and 100 per cent.




Table 2 Standard errors for percentages obtained from sample data

95 per cent Standard Confidence

Sample error interval size Percentage (% age) (% age)

10 10 9 5 0-29 10 50 15 8 18-82

25 10 60 0-22 25 50 10 0 30-70

100 10 3 0 4-16 100 50 5 0 40-60

400 10 1 5 7-13 400 50 25 45-55

Table 2 shows the standard errors for particular percentages (10 per cent and 50 per cent) and different sample sizes. For example, for a percentage value of 50 per cent based on a sample size of 25 the user can be confident 95 per cent of the time that the true percentage figure is in the range 30 per cent to 70 per cent (that is, 50 ? 2SE).

Conclusion

Census users frequently fail to appreciate that the 1991 SAS are a complex set of statistics. It is essential that any prospective user of the 1991 SAS should be aware of the potential impact that data suppression, data modification, variations in population size imputation, and sampling error can have on the reliability of any results based upon an analysis of the SAS at ED/OA level.

Acknowledgements This paper was presented in the session ' Small area population change in the UK 1981-91 at the Annual IBG Conference, Royal Holloway and Bedford New College, Wednesday 6th January 1993. The work was funded by the ESRC (project H507 26 5024) as part of the ESRC/JISC 1991 Census of Population Initiative. I would also like to thank my colleagues John Roberts, Steve Simpson, Virginia Knight and Malcolm Campbell for their helpful advice.

References Butcher B and Elliot D (1986) A sampling errors manual (OPCS, London)

Clark A M and Thomas F G (1990) 'The geography of the 1991 Census' Population Trends 60, 9-14 Cole (1993) 'The 1991 Local Base and Small Area Statistics' in Dale A and Marsh C (eds) The 1991

Census User's Guide (HMSO, London) CRU/OPCS/GRO(S) (1980) People in Britain-a census atlas (HMSO, London)

DoE (1983) 'Urban Deprivation' Information Note 2 Inner Cities Directorate, Department of

Environment, 2 Marsham Street, London HMSO (1992) 1991 Census Definitions Volume (HMSO, London) Holterman S (1975) ' Areas of urban deprivation in Great Britain: an analysis of 1971 census data' Social

Trends 6, 33-47



78 Cole

Jones K and Kirby A (1980) 'The use of chi-square maps in the analysis of census data' Geoforum 11,

409-417 Jones K and Moon G (1987) Health, Disease and Society (Routledge, London)

Morphet C (1992) 'The interpretation of small area census data' Area 24, 63-72

OPCS (1983) Census Monitor, CEN 83/6 (OPCS, London) OPCS (1985) Census Monitor, CEN 85/1 (OPCS, London) Rhind D (1983) (ed) A census user's handbook (Methuen, London) Senior M (1991) ' Deprivation payments to GPs: not what the doctor ordered ' Environment and Planning

C 9, 79-94

Visvalingham M (1978) 'The signed chi-square measure for mapping' Cartographic Journal 15, 93-8

Weis N A and Hassett M J (1991) Introductory statistics third edition (Addison-Wesley)



data modification, data suppression, small populations and other features of the 1991 small area...

Documents