MA Quantitative Methods

Official Statistics, Censuses and Sociology

Peter Ratcliffe

Lecture outline

• What are ‘official statistics’?• What questions are raised by their

generation and use? – ethical, political, methodological

• Why are some (UK) sociologists loathe to use official statistics?

• General issues raised by ‘secondary data’• CASE STUDY: Population Census in

Britain

OFFICIAL STATISTICS

In Britain: many sources of such data…..

• Vital registrations – basic demographic data• Intermittent surveys (Labour Force Survey,

General Household Survey, English House Condition Survey, British Crime Survey, etc.).

• One-off, ad hoc, surveys, and probably most important of all

• The decennial Census of Population

SECONDARY DATA: Key Issues

Essentially two-fold:

• Ethical and Political

• Methodological

Ethical and Political IssuesData don’t just emerge from the ether – they are produced by

conscious social actors deciding what data are ‘needed’

Questions raised by this?

• What is meant by ‘needed’ – by whom and why? • Do some not want the data in question? Why are data on some

issues available and not on others?• Why are concepts measured in a particular way? (e.g. definition of

unemployment)• The latter issue is of particular importance for the secondary

analyst because of the need to assess the theory/value-ladenness of data.

• Are you coming into the analysis with a radically different perspective from that of the original researcher?

• Is this such that the data are of little use/interest to you?

Methodological concerns

Tim May in his book Social Research (2001) – argues that these concerns depend on the theoretical position adopted:

• Positivist• Institutionalist• Radical

Positivist

In favour of the use of such data, subject to a number of conditions being satisfied:

• Reliability

• Validity - internal v. external [e.g. income]

• Sampling Error

Institutionalist Focus is on the difference between measurement in

the social sciences as against the natural sciences. [Actors, may well ‘see’/interpret events in different ways.]

Therefore, key question is: are the statistics concerned really an aggregation of ‘the same thing’?

EXAMPLES:• Suicide – who decides and how?• Crime Statistics – e.g. racist attacks, drug taking,

motoring offences………….

Radical Statistics tell us more about the organisation

collecting/compiling the statistics than they do about the social world they purport to describe.

Thus………an extension of earlier point that

statistics are social constructs emanating from those who devise them (in this case, ‘the state’/employees of government departments)

But, in terms of ‘the state’, is this necessarily so?

Certainly true that, with few exceptions, the data are not primarily collected for our benefit. But….

Why do UK sociologists often shun official statistics?

• What Americans call ‘math phobia’?

• We can’t measure the social world – the data are essentially the result of a social process that distorts social reality.

• Data are an artifact of the beast which collect them - therefore of no use to us.

• Problems of reliability and validity (or, indeed, sampling error)

Secondary analysis: key issues

Angela Dale in Doing Secondary Analysis (1988):

• What was the purpose of the study? Was it an academic study designed to explore background issues? Was it a quick poll aimed at capturing attitudes at one point in time? What was the conceptual framework that informed the study?

• What information has been collected? Does it

cover the range of issues in which the researcher is interested? What categories have been used for classifying, say, occupation or marital status? Do the data incorporate the distinctions required by the secondary analyst?

Secondary analysis: key issues

• What sampling frame was used, and what is the sampling unit – that is, has the study sampled individuals, or households, or employers? What are the potential biases in the data? What is the response rate?

• Who was responsible for collecting the data? What is the quality of the data?

• (If a survey) Are the data nationally representative? Will they support generalisations about the population sampled? Are any weighting procedures needed?

• When were the data collected? Are they still relevant, or have there been substantial changes that make the data source of little value?

Secondary analysis: benefits

• Access to much larger samples than one would normally have

• No longer reliant on published output/tables• Clearly. quicker than launching your own study• Ethical gains – non-obtrusive. Also, the re-

use/re-analysis of data may render a further study unnecessary.

Now, a important case study……………

Population Census, England & Wales:historical background

• Census every ten years since 1801 except in times of war, e.g. 1941. [Very occasionally held more often (e.g. 1966)]

Why was a census first seen as desirable?

Latter part of the 18th Century (influence of Enlightenment?), certain politicians had argued that these were needed:

• In order to plan (control?) the development of British society• To satisfy general needs• Measure population growth – to assess potential military

power as well as food needs.

[Other politicians argued that the very idea of a Census was an outrageous invasion of privacy.]

Population Census, England & Wales:historical background

Importantly, though - ‘health’ of society was also interpreted in ethnic and class terms, e.g. characterisation of the Irish

Context - development of Social Darwinism and the Eugenics movement

The agenda has changed markedly over time (along with broader changes in society)……

Population Census: England & Wales

Ethical/Political and other agenda issues:[discussed further in this week’s seminars]

• Should we be asking, for example, about ‘race’/ethnicity and religion in the national census?

• Are there important questions that are

omitted? Material on gender issues?

Methodological concerns• Reliability? Training of census enumerators. Also,

post-enumeration study (PES)• Validity? Can a particular set of data generate a

valid measure of what the researcher wants? (Number of rooms as a surrogate measure of space? Measures of health status? What about the ‘ethnic group’ question?)

• Sampling Error? Even in a Census? Actually there is a small amount of non-response – around 2%. Significance?

• Data comparability – key issue for the analysis of social change or formal longitudinal analyses

Measurement of ‘ethnic group’: ethical/political questions

• Why are such data wanted, and by whom? Are there dangers in such data being collected and made available to social scientists and others?

Historical background to the question:

• Place of birth no longer a useful surrogate for ethnicity

• Need to assess the effectiveness of ‘Race Relations’ legislation

• Needed to tailor service delivery to different communities (?)

• Why was it excluded from the 1981 Census?

Measurement of ‘ethnic group’: methodological issues

Questions for this week’s seminars:

• What is the question measuring?• Is it really about ‘ethnic group’? Is it not about

‘race’?• What are the key differences between the 2001

variant and the previous version? • Is this new question ‘better’ – in what sense(s)?

What about the 2011 version?• What are the challenges faced by those interested

in change processes via longitudinal analysis? [Comparability issues?]

Measurement of Social Class

Social Class – measured by ‘Economic Activity/Occupation’ (how well are gender issues incorporated within census questions?)

Presentation of ‘social class’ data shifted after 2000…..

Register General’s Social Class and Socio-Economic Group have been replaced by NS-SEC (National Statistics – Socio-Economic Classification). Check this out at: http://www.ons.gov.uk/ons/guide-method/classifications/archived-standard-classifications/ns-sec/continuity-issues--sc--seg-and-ns-sec/index.html This gives you a mapping of the former onto the latter.

The publication of census data

The census generates a large amount of data on:

• Demography (age, sex, household structure, etc.)• Economic status and occupation • Educational qualifications, and • Housing

This is then published in printed and electronic form at a variety of spatial scales: e.g. GB, country, Registrar General’s Standard Regions, counties/districts, wards, and finally enumeration districts. There are also ‘Small Area Statistics’.

Samples of Anonymised Records (SARs)

Problems – researcher has no control over data analysis. Can pay for ‘special tabulations’ – but……..

Researchers therefore lobbied over many years for samples of data – known in the US as ‘public use samples’.

OPCS (now ONS) finally relented in 1991 with the introduction of Samples of Anonymised Records (SARs).

• 1% sample of households, and • 2% sample of individuals

[Certain restrictions on access and a low level of spatial detail (for ethical reasons - confidentiality and anonymity). And data often subject to ‘Barnardisation’.]

Conclusions

• Official statistics contain a wealth of important data on the social world

• May add significantly to multi-method research (including your projects!)

• Researchers need to exercise care when using them

• BUT…..this should not be seen as a reason for rejecting their use