european health and social integration survey...list of countries 6 eu comparative quality and...
Post on 24-Jun-2020
7 Views
Preview:
TRANSCRIPT
European Health and Social Integration Survey EU comparative quality report June 2015
Table of contents
3 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
List of abbreviations .................................................................................................................. 5 List of countries ......................................................................................................................... 6
1. INTRODUCTION ..................................................................................................................................... 7 1.1 Survey Coverage and Timing ............................................................................................. 7 1.2 Assessing Cross-National Survey Quality .......................................................................... 8
2. BACKGROUND AND METHODOLOGY .................................................................................................. 9 2.1 Cross-national Harmonisation ............................................................................................ 9 2.2 Basic Concepts ................................................................................................................. 10
2.2.1 Measuring Disability in EHSIS ................................................................................... 10 2.2.2 Other Measures of Disability ...................................................................................... 11 2.2.3 Measuring the Severity of Disability in EHSIS ........................................................... 11
3. QUALITY CONTROL ............................................................................................................................. 12 3.1 Contractors’ Overall Quality Assessment ......................................................................... 12
3.1.1 EU Core Social Variables .......................................................................................... 13 3.1.2 Lessons Learned and Contractors’ Suggested Improvements ................................. 13
4. PARTICIPATION ................................................................................................................................... 15 4.1 Countries Taking Part ....................................................................................................... 15 4.2 Survey Population ............................................................................................................. 16 4.3 Data Collection Period ...................................................................................................... 16 4.4 Survey Reference Period .................................................................................................. 17 4.5 Voluntary Nature of the Survey ......................................................................................... 17
5. ACCURACY ........................................................................................................................................... 18 5.1 Sample Design .................................................................................................................. 18
5.1.1 Sample Size ............................................................................................................... 19 5.1.2 Substitution ................................................................................................................ 20
5.2 Sampling Errors ................................................................................................................ 20 5.3 Nonsampling Errors .......................................................................................................... 24
5.3.1 Coverage Error .......................................................................................................... 24 5.3.2 Nonresponse .............................................................................................................. 24
5.3.2.1 Imputation ........................................................................................................... 28 5.3.3 Measurement Error .................................................................................................... 28
5.3.3.1 Data Collection Mode ......................................................................................... 28 5.3.3.2 The Questionnaire .............................................................................................. 30 5.3.3.3 The Interviewer ................................................................................................... 34 5.3.3.4 The Respondent ................................................................................................. 38
5.3.4 Data Processing Error ............................................................................................... 39 5.3.4.1 Data Entry, Coding and Cleaning ....................................................................... 39 5.3.4.2 Weighting Adjustments ....................................................................................... 42
6. REFERENCES ...................................................................................................................................... 46 7. ANNEXES .............................................................................................................................................. 48
Annex A .................................................................................................................................. 48 Annex B .................................................................................................................................. 52 Annex C .................................................................................................................................. 72
Table of contents
4 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
Figures Figure 5.1: Response Rates by Mode and Country ........................................................................... 25
Tables Table 1.1: Contractual Wave by Country ............................................................................................... 7
Table 1.1: Contractual Wave by Country (continued) .......................................................................... 8
Table 1.2: Common Dimensions of a Survey Quality Framework ..................................................... 8
Table 2.1: Overall Measure of Disability .............................................................................................. 10
Table 4.1: Countries Administering the Survey ................................................................................... 15
Table 4.2: Data Collection Period ......................................................................................................... 16
Table 5.1: Sampling Frame, Unit and Design by Country and Mode of Data Collection .............. 19
Table 5.2: Minimum Required and Achieved Sample Size by Country ........................................... 21
Table 5.3: Proportions, Standard Errors and Confidence Intervals for Six Key Disability Measures by Country .............................................................................................................................. 22
Table 5.4: Percent of Nonrespondents in Each Country by Reason for Nonresponse ................. 26
Table 5.5: Average, Minimum and Maximum Item Nonresponse Rates by Country ..................... 27
Table 5.6: Mode of Data Collection by Country .................................................................................. 29
Table 5.7: Languages Used to Conduct the Survey by Country ...................................................... 30
Table 5.8: Interview Duration by Mode of Administration .................................................................. 33
Table 5.9: Respondent/Interviewer Ratio by Country and Mode of Administration ....................... 36
Table 5.10: Mean Number of Contact Attempts by Country and Mode of Administration ............ 37
Table 5.11: Percent Proxy Interviews by Country ............................................................................... 38
Table 5.12: Data Processing Errors/Issues Detected by Country and Mode of Administration .. 41
Table 5.13: Weighting Procedure by Country ..................................................................................... 43
Table A-1: Annex A: Questionnaire Guide ........................................................................................... 48
Table B-1: Comparison of Population Percentages to Weighted and Unweighted Sample Estimates of Demographic Characteristics by Country ...................................................................... 52
Table C-1: Question Modifications and Problems by Country .......................................................... 72
List of abbreviations
5 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
List of abbreviations CAPI Computer-Assisted Personal Interviewing
CATI Computer-Assisted Telephone Interviewing
CAWI Computer-Assisted Web Interviewing
EHIS European Health Interview Survey
EHSIS European Health and Social Integration Survey
ESS European Statistical System
EU European Union
ICF International Classification of Functioning, Disability and Health
ITT Invitation To Tender
ISCED International Standard Classification of Education
ISCO International Standard Classification of Occupations
LFS Labour Force Survey
MEHM Mini-European Health Module
MS Member State
NR Nonresponse
PAPI Paper-And-Pencil Interviewing
TSE Total Survey Error
TSQ Total Survey Quality
RDD Random Digit Dialing
List of countries
6 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
List of countries AT Austria
BE Belgium
BG Bulgaria
CZ Czech Republic
DK Denmark
DE Germany
EE Estonia
EL Greece
ES Spain
FR France
IT Italy
CY Cyprus
LV Latvia
LT Lithuania
LU Luxembourg
HU Hungary
IS Iceland
MT Malta
NL Netherlands
NO Norway
PL Poland
PT Portugal
RO Romania
SI Slovenia
SK Slovakia
FI Finland
SE Sweden
UK United Kingdom
Introduction
7 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
1
Acknowledgements This publication is the outcome of coordinated efforts by Pascal Wolff and Lucian Agafitei.
SOGETI Luxembourg SA supported the work, the team being composed of Laurent Jacquet (project manager), Julien Arcade (statisticians and SAS programmers). Furthermore, methodological support was provided by Amanda Wilmot and Howard Meltzer (University of Leicester).
1. INTRODUCTION This report provides a summary of the quality and technical aspects of the European Health and Social Integration Survey (EHSIS). The EHSIS is a cross-national survey carried out for the first time in 28 European countries in 2012/13 with the goal of providing comparable cross-national data on disability.
The design of EHSIS was different from most other disability surveys. Taking account of the International Classification of Functioning Disability and Health (ICF) conceptual approach to disability, questions were designed to measure the biopsychosocial model of disability, to take account of the person, their impairments, activity limitations and the environment and the interrelationships between them. The concepts used to measure disability in this way are described further in section 2.
1.1 Survey Coverage and Timing The survey was tendered in 2011 by Eurostat (2011/S 122-201426 and 2011/S 249-404645). The invitation was extended to the 27 European Member States1 (MS) plus Iceland and Norway.
The contract for Ireland was cancelled and, therefore, the survey in this country did not take place. Survey preparation, administration and delivery of final tables along with each country’s quality and technical reports took place between December 2011 and April 2014. The contracts were let in two waves. Table 1.1 shows which countries conducted the survey according to each wave. The first wave was tendered to begin December 2011 for a 21-month duration; the second wave was tendered to begin April 2012 for a 17-month duration. In the second wave, the field work period was extended to maximum 6 months. Spain run the survey without participating to the above-mentioned calls for tenders.
Table 1.1: Contractual Wave by Country
Wave Country Wave Country
1 Bulgaria 2 Austria
Denmark Belgium
Estonia Cyprus
France Czech Republic
Germany Finland
Greece Iceland
Hungary Italy
(continued)
1 Croatia was not a Member State at the moment of launching the call for tender
1 Introduction
8 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
Table 1.1: Contractual Wave by Country (continued)
Wave Country Wave Country
Latvia Luxembourg
Lithuania Netherlands
Malta Norway
Poland Portugal
Romania Sweden
Slovak Republic
Slovenia
United Kingdom
1.2 Assessing Cross-National Survey Quality The aim of this quality report is to provide information on how well the survey satisfies the goals for each quality component. In order to facilitate the reporting of survey quality across countries, guidelines are provided by the European Statistical System (ESS) based on a standardized Quality Framework. In this report, output quality is assessed in line with the ESS Quality Framework Guidelines and is reported in terms of the components in Table 1.2.
Table 1.2: Common Dimensions of a Survey Quality Framework
Accuracy Total survey error is minimized.
Credibility Data are considered trustworthy by the survey community.
Comparability Demographic, spatial and temporal comparisons are valid.
Usability/Interpretability Documentation is clear and metadata are well managed.
Relevance Data satisfy user needs.
Timeliness/Punctuality Data deliveries adhere to schedules.
Completeness Data are rich enough to satisfy the analysis objectives without undue burden on respondents.
Coherence Estimates from different sources can be reliably combined.
Source: Biemer, P. (2010). Total survey error. In M. Lovric (Ed.), International encyclopedia of statistical science. New York: Springer.
It is worth noting that, when assessing total survey quality, a fitness-for-use concept is incorporated that includes, relevance, usability and timeliness of the data production. The report is based on the ESS guidelines. However, it is reliant on the information provided by each individual country conducting the survey. Each in-country contractor was required to provide a quality report based on a quality reporting template provided by Eurostat. All of the described quality components are covered. This report also documents the methodology as well as the administration of the survey, thus fulfilling the function of a technical report as well as a quality report. This report will describe the process of conducting the survey and comment on the survey quality components therein with the aim of highlighting any shortfall in quality that may have affected survey estimates.
National quality and technical reports delivered to Eurostat constitute the source for the present report
Background and methodology
9 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
2
2. BACKGROUND AND METHODOLOGY 2.1 Cross-national Harmonisation In order to facilitate cross-national comparison using one data collection instrument, a decision was taken to provide harmonised inputs for EHSIS by providing a model questionnaire. The questionnaire provided specified question wording that had been previously tested across a number of European countries, both through quantitative pilot study and qualitative cognitive pretesting. The development of the survey questionnaire was iterative and comprised an initial wave of testing and reviews with three Member States: United Kingdom, Italy and Lithuania (Meltzer 2008). A second wave of testing and review was carried out in 10 Member States involving: Bulgaria, the Czech Republic, Estonia, Finland, Greece, Hungary, Latvia, Malta, Slovakia and Spain. (Wilmot and Meltzer 2010).
In addition to attempting to harmonise the questionnaire, and in order to further control for other variants when administering a survey cross-nationally, the survey specification required certain elements to be functionally harmonised whilst being adaptive to specific country situations. Eurostat provided additional technical specifications to be followed. These included: the timing of the data collection; specified target population; sample selection including minimum sample size; mode of administration (2) ; the data collection instrument and interviewer instructions (3), translation protocol and pretesting requirements; the technique and organisation of the interviews; data processing; validation and weighting rules, and the way in which the data were to be delivered. Each is covered in this report. The introduction to each section of the report will describe the relevant specification requirement.
Survey errors can arise throughout the statistical value chain. Although the aim of the survey administration is, of course, to maximize the precision of measurement, there will undoubtedly be sources of survey error that affect that precision. While some sources of error can be statistically controlled for, others cannot be quantified or controlled and therefore, in such instances, a qualitative assessment of those errors, such as translation issues or adaptions made to the questionnaire, is made or simply described.
(2) It should be noted that although the EHSIS questionnaire was originally designed to be interviewer-administered face-to-face, the tender specifications, although strongly recommending this mode of data collection, also allowed for telephone or web-based data collection.
(3) The classificatory questions were also prespecified and included the 2011 EU core social variables, although it should be noted that these questions are specified according to harmonized outputs, rather than inputs. CORE VARIABLES: UPDATED 2011 GUIDELINES Doc. Eurostat/F/11/DSS/01/2.3EN, MEETING OF THE EUROPEAN DIRECTORS OF SOCIAL STATISTICSLUXEMBOURG, 21-22 SEPTEMBER 2011
2 Background and methodology
10 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
2.2 Basic Concepts 2.2.1 Measuring Disability in EHSIS EHSIS was designed to measure the biopsychosocial model of disability. This is not the same as measuring disability according to the more traditional medical model, which locates the problem within the individual. According to the biopsychosocial model applied to the survey, people with disabilities are those who face barriers to participation in any of 10 domain activities from the ICF identified as being of most relevance, associated inter alia with a health problem or basic activity limitation.
Each section of the questionnaire corresponds to one of the 10 life domains identified: mobility; transport; accessibility to buildings; education and training; employment; the internet; social contact and support; leisure pursuits; economic life, and the attitudes and behaviour of others. Each section of the questionnaire examines different facets of life that enable an individual to be a fully functional and integrated member of society. Each section covers disadvantage or restrictions to social participation that people, with and without a health problem, face in their everyday lives. Hence, information is collected on the experience of barriers in each of the 10 life domains.
Owing to the constraints of producing a questionnaire of reasonable length, plus the need for questions to apply to everyone across Europe, the decision was made to sacrifice level of detail in order to cover the variety of social circumstances. Each section starts with a question aimed at identifying any barriers to social integration identified by survey respondents. The section continues by looking at the contribution of health conditions or basic activity limitation to that experience. The reasons people have problems in social integration can be a mixture of personal factors, environmental factors, as well as health conditions or difficulties with basic activities; and in many cases they operate simultaneously. To try to reflect real life, the questions on barriers list these factors, and respondents could rate as many items they feel contribute to their lack of social integration in each domain. A person would be categorised as disabled if he or she selected a health condition or activity limitation at any of the barrier questions in each of the 10 life domains included in the survey (Meltzer 2011) —see Table 2.1. The questions focused on barriers to participation, as opposed to facilitators that may already have been put in place and that people may not be aware of or have taken for granted.
Table 2.1: Overall Measure of Disability
Section Variable description
Mobility Barriers to mobility
Transport Barriers to using private vehicle
Transport Barriers to using other forms of transport
Accessibility to buildings Barriers to accessing buildings
Education and Training Barriers to education and training
Employment Barriers to employment
Internet use Barriers to using the internet
Social contact and support Barriers to speaking with people feel close to
Leisure pursuits Barriers to pursuing hobbies or interests
Leisure pursuits Barriers to attending cultural events
Economic life Barriers to paying for the essential things in life
Attitudes and behaviour Reasons for feeling treated unfairly
Background and methodology
11 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
2
2.2.2 Other Measures of Disability In addition to the EHSIS measure of disability described above, the survey questionnaire is supplemented with other measures of disability used in cross-national European surveys namely: the three MEHM (4) questions on self-perceived health, chronic conditions and activity limitation, used in surveys such as the European Health Interview Survey (EHIS) and the European Union Statistics on Income and Living Conditions (EU-SILC); questions from the Labour Force Survey 2011 Ad Hoc Module on the employment of disabled people that presented a list of health problems and basic activity limitation questions: seeing, hearing, walking etc.; as well as questions used in the EHIS on activities of daily living (ADL): feeding oneself, dressing, bathing etc.; and Instrumental Activities of Daily Living (IADL): preparing meals, shopping, light housework etc. The consequence of adding these extra questions is that alternative definitions of disability can be derived and cross-tabulations made with the EHSIS definition (see Meltzer et al. 2010).
2.2.3 Measuring the Severity of Disability in EHSIS Several measures of severity can be derived from the survey. For the biopsychosocial model, severity of disability would have to take account of whether a health problem or a basic activity limitation was a contributory factor in the lack of participation. So only participation restrictions linked to health problems or impairments are included in the measure.
A further refinement can be added by looking at the supplementary questions asked of those with a participation restriction related to a health problem who also have a lack of special equipment and/or personal assistance. This has a great advantage over questions which just ask respondents to report a degree of difficulty as such distinctions can be interpreted in different ways.
Therefore, one can have four levels of severity based on these questions.
- Lacks no specialised equipment or personal assistance - Lacks specialised special aids or equipment - Lacks personal help or assistance - Lacks specialised equipment and personal assistance
Another way for measuring severity is by adding up the number of life domains where a person encounters a barrier associated with a health problem or basic activity limitation.
(4) Mini European Health Module
3 Quality control
12 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
3. QUALITY CONTROL 3.1 Contractors’ Overall Quality Assessment One section of the quality reporting form asked each country contractor to provide an overall assessment of the strengths and weaknesses of the survey; these assessments are discussed in this section. (Note that some quotes are provided in italics to illustrate and support the points discussed.)
Overall contractors reported positively, both on the aims of the survey and on the overall quality of the survey and its administration.
“The strongest point about the EHSIS is its set-up and existence: it deserves a positive assessment that the EHSIS attempts to close the obvious data and information gaps…in context of the international monitoring requirements of the UN Convention on the Rights of Persons with Disabilities and the European Disability Strategy. “ [Germany]
“The strengths of the survey are: the purpose and value of the project.” [Lithuania]
“Overall, the survey was of high quality, and very few issues were experienced.” [Iceland]
“The EHSIS project was prepared and conducted very precisely with big attention on the details. The general organisation of the project was at a very high level.” [Bulgaria]
A number of countries commented that both interviewers and respondents recognised the importance of the subject matter.
“The subject of the survey is generally perceived as important and worthwhile for both interviewers and respondents.” [Norway]
“The survey was clearly important to respondents; those who agreed to participate were keen to do so.” [Finland]
“The respondents found the content of the survey as very interesting and they did not report on difficulties answering the questions.” [Slovenia]
The questionnaire and interviewer instructions were said to be clear and well structured. In general, survey respondents were able to comprehend and respond to the questions asked.
“Questionnaire handled a potentially sensitive issues—disability—in a very appropriate manner.” [Netherlands]
“We were impressed by the quality of the questionnaire.” [Poland]
“Strong points—questions were easy to understand and answer.” [Cyprus]
One of the main weaknesses of the survey was said to be the length of the questionnaire, which also seemed repetitive to those respondents who did not report any health problems or activity limitations.
“Weakness points—length of interview.” [Cyprus]
“The respondents found the questionnaire very long indeed.” [Denmark]
“Some respondents especially those without any physical disabilities, found the questions to be rather repetitive. However, we understand the necessity of this questioning technique, and so would not recommend any changes.” [Sweden]
The repetitive nature of the questionnaire may have been more apparent for those taking part in a telephone interview. The questionnaire was originally designed to be interviewer-administered face-to-face, either CAPI or PAPI. It is recommended that a questionnaire designed for telephone or web administration should be prepared for any future rounds of the survey in order to help ensure comparability across modes. See Table 5.7 for mode of data collection.
Quality control
13 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
3
“The quality of the survey is good for personal interviews. When it comes to telephone and web interviews we are uncertain of the effect on quality.” [Denmark]
“This questionnaire is designed better for CAPI or CAWI interviews that for CATI interviews…the interviewer has to read all of the options of the ‘life domain question’ that are very repetitive and it causes fatigues for the interviewed people.” [Spain]
3.1.1 EU Core Social Variables The requirement for the use of the EU core social variables, which are harmonised on outputs rather than inputs, caused some problems at the development stage of the survey. Contractors were unsure of question wording or had difficulties mapping outputs to the international classifications used, although a list of in-country coordinators/advisers for the EU core social variables, located within the respective National Statistical Institutes, was provided by Eurostat. Contractors asked for the actual question wording for the core social variables for any future rounds of the survey.
3.1.2 Lessons Learned and Contractors’ Suggested Improvements The main issue was conducting the survey in modes other than face-to-face. Because each response option was read aloud by telephone interviewers (that would otherwise have been shown using a card), contractors administering the survey by telephone reported that the interviews were long and repetitive, which could have led to respondents terminating the interview.
“The questionnaire was thoroughly tested in advance, but it was not tested for other methods than face-to-face interviewing.” [Denmark]
“Hard to keep respondents attention due to repetition of questions, Respondents get a feeling of the “the same over the same” and get tired.” [Lithuania]
“From our point of view, the questionnaire structure was too reiterative and tedious.” [Luxembourg]
Some of the contractors said that they would have preferred a longer field period in order to accommodate the complications of collecting data during the winter. In the Czech Republic, for example, performing data collection during a long and difficult winter season made face-to-face survey administration more difficult. Another contractor stated:
“The cold winter (with short days) made fieldwork even more difficult; it would be better to carry out this type of study in late Spring/early Summer or very early in the Autumn.” [Italy]
Overall, contractors recommended a longer field period.
“If possible the field period should be longer than 6 months.” [Finland]
“Improvement could have been done if the field work would be longer.” [France]
The length of the data collection period was reported as having affected survey response rates. It was suggested that Eurostat could have provided further guidance on maintaining response rates and avoiding refusals from respondents. Italy suggested including the Eurostat letter head in the advance letter and other respondent materials to improve the response to the survey. Eurostat was mentioned in the suggested text for the advance letter. However, Eurostat had no evidence that using its logo helps with response.
It was also suggested that substitutions should be allowed where a field period is restricted. Indeed, Bulgaria and Lithuania reported that sample substitution had been permitted—but with no further
3 Quality control
14 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
explanation. Cyprus selected additional Primary Sampling Units “due to the fact that some roads had no houses or high nonresponse rate in some areas (i.e., high class areas) and areas with foreigners.” See section 5.1 sample design.
Another issue related to the section on employment and education was that additional explanation was required when the question was asked of older retired people; they felt these sections did not apply to them.
The United Kingdom, a CATI survey, expressed concern over the length of the introduction and therefore allowed interviewers to shorten the introduction by explaining who the survey was for at the end and not at the start, as stipulated.
One of the issues that caused problems for contractors was in applying the correct weights.
Participation
15 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
4
4. PARTICIPATION 4.1 Countries Taking Part Twenty-eight countries took part in the survey as shown in Table 4.1: 26 Member States of the European Union (5), plus Iceland and Norway.
Table 4.1: Countries Administering the Survey Country Country Abbreviation
Austria AT
Belgium BE
Bulgaria BG
Cyprus CY
Czech Republic CZ
Denmark DK
Estonia EE
Finland FI
France FR
Germany DE
Greece EL
Hungary HU
Iceland IS
Italy IT
Latvia LV
Lithuania LT
Luxembourg LU
Malta MT
Netherlands NL
Norway NO
Poland PL
Portugal PT
Romania RO
Slovak Republic SK
Slovenia SI
Spain ES
Sweden SE
United Kingdom UK
(5) The survey was not conducted in Croatia and Ireland.
4 Participation
16 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
4.2 Survey Population In each participating country the target population was all those aged 15 or over living in private households. In France the target population was described as individuals aged 15 or over living in private households and resident in metropolitan France. In Malta the territory covered included Malta and Gozo, while in Portugal the target population was described as “individuals aged 15 or over living in private households in localities with 10 or more dwellings, living in Mainland Portugal.”
In each country the survey was administered to only one person in a household.
4.3 Data Collection Period The length of the field period can affect response rates, as commented on in some national reports. For most countries, field work took place during autumn 2012 and beginning of 2013. Table 4.2 shows the data collection periods for each country taking part in the survey. Data collection lasted from 1.5 months (Hungary) to 8 months (Portugal).
Table 4.2: Data Collection Period
Country Data Collection Period
AT 8 October 2012–28 February 2013
BE 18 September 2012–15 January 2013
BG 1 September 2012–30 November 2012
CY 15 October 2012–10 January 2013
CZ 16 November 2012–12 April 2013
DK 6 August 2012–10 October 2012
EE 14 September 2012—3 December 2012
FI 14 September 2012–26 March 2013
FR 7 September 2012 -20 December 2012
DE 15 August 2012 -3 0 November 2012
EL 7 September 2012-13 December 2012
HU 15 September 2012-30 October 2012
IS 18 September2012–28 November 2012
IT 1 February 2013–3 June 2013
LV 1 September 2012 to 9 December 2012
LT 5 September 2012—16 December 2012
LU 23 October 2012–28 February 2012
MT 2 September 2012 to 30 November 2012
NL 20 September 2012–11 January 2013
(continued)
Participation
17 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
4
Table 4.2: Data Collection Period (continued)
Country Data Collection Period
NO 5 November 2012—18 February 2013
PL 6 September 2012–12 December 2012
PT 5 November 2012—9 July 2013
RO 15 October 2012–20 December 2012
SK 1 September 2012—30 November 2012
SI 24 August 2012–15 December 2012
ES 24 September 2012 to 13 March 2013
SE 21 September 2012–9 January 2013
UK 13 September 2012–3 February 2013
4.4 Survey Reference Period There were no specific reference periods for the survey. The survey questions asked about the respondent’s current situation at the time of interview.
4.5 Voluntary Nature of the Survey In all countries, respondent participation in the survey was voluntary.
5 Accuracy
18 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
5. ACCURACY Surveys collect information about a specific population and report estimates of unknown population characteristics. Such estimates may or may not match exactly the unknown true population values. The accuracy of such estimates may be affected by various errors of different magnitude and from different sources that can potentially bias the estimates or contribute to greater variability from the true values. We divide this section into two major subsections, based on error sources—sampling errors and nonsampling errors. We summarize the methodological challenges across countries in dealing with these sources of error and what has been done to minimize and adjust for them.
5.1 Sample Design The tender specifications stated that the sampling design should be based on a probability sampling method that ensures accurate and representative results for the whole population within that country. Based on information provided in the technical and quality reports, all countries used probability-based samples. The target population for all countries was defined as individuals 15 years of age or older, living in private households. One eligible member per household was selected in each country. When the sampling frame was at the household, rather than person level, the Kish or last birthday methods were used for selecting the respondent.
Even though the recommended mode for the EHSIS was face-to-face, many countries employed different modes; thus, a variety of sampling designs were used. A majority of countries employed some kind of multi-stage stratified design. Table 5.1 summarizes the sampling design used in each country by mode of data collection. The most common design for face-to-face modes was stratified cluster sampling, implemented in eight countries; while the most common sampling design for the telephone mode was stratified random sample, used in all but one of the 14 countries conducting the survey via telephone.
Seven countries (CY, FI, HU, LV, NO, PL and PT) oversampled the youngest members of the population (15-24 years old) under the premises that the disability prevalence in such subpopulations is lower and overrepresentation is needed for having reliable data (6).
A variety of sampling frames were employed across countries. In all countries the frames were as exhaustive as possible with respect to the target population and in majority of the countries, all sampling units had a known, non-zero probability of selection (7). Population registers were used in 11 countries, random digit dialing (RDD) and list-assisted RDD was used in 10 countries, and area probability samples were drawn in the remaining seven countries. In 18 countries, the sampling frame was updated during the year of the survey data collection (2012); in six countries (CY, CZ, IT, LT, MT, and ES) the frame was a year older. An older frame (8) (often supplemented by 2011 census data) was used in three countries (EL, PT, RO). Only EE indicated ‘no applicable’ for the year of the sampling frame.
6 Oversampling of young population was recommended in the ITT 7 Several countries (Bulgaria, Cyprus, Greece, Portugal and Lithuania) used the random walk approach within each PSU. In Bulgaria,
probabilities of selection were not calculated even at the within-household level. In Cyprus, Greece and Lithuania, within household probabilities of selection were calculated. Portugal’s sampling weight included probabilities of selection at the PSU, housing unit and within-household level.
8 Frames dated between 2001-2006
Accuracy
19 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
5
Table 5.1: Sampling Frame, Unit and Design by Country and Mode of Data Collection
Mode Sampling Frame Sampling Unit Sampling Design Country
Face-to-face
Area probability Address Stratified cluster sample BG, CY, CZ, EL, IT, LT, PT
Population register Address Stratified random sample PL
Population register Address Stratified systematic sample
RO
Population register Individual Stratified random sample FI, HU, SI
Population register Individual Stratified cluster sample LV (a)
Population register Individual Stratified systematic sample
MT
Telephone List (landline)
List-assisted RDD (mobile)
Household (landline)
Individual (mobile)
Stratified random sample BE, NL, SE
List (landline)
RDD (mobile)
Household (landline)
Individual (mobile)
Stratified random sample SK
List-assisted RDD (landline)
RDD (mobile)
Household (landline)
Individual (mobile)
Stratified random sample (landline)
Systematic sample (mobile)
UK
RDD Household (landline)
Individual (mobile)
Stratified random sample AT, DE
RDD Household (landline)
Individual (mobile)
Simple random sample EE, FR
RDD Household Simple random sample LU
Population register Address Stratified random sample ES (b)
Population register Household (landline)
Individual (mobile)
Stratified random sample NO
Population register Individual Systematic sample DK (c)
Population register Individual Stratified random sample IS
(a) Data collection methods also included CATI (b) Web option available
(c) Web option for individuals with unknown phone numbers
5.1.1 Sample Size
5 Accuracy
20 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
Three definitions of sample size are of interest—the minimum required sample size, the achieved sample size (number of completed interviews), and the effective sample size (the achieved sample size, divided by the design effect (9)). The tender specifications included a minimum required sample size for each country. Table 5.2 presents the required and achieved sample size by country. The effective sample size is not included because most countries did not report design effects.
Assuming a design effect (10) of 1, all countries achieved or exceeded the required minimum sample size. However, this information should not be taken for granted, as most countries employed complex sample designs and weighting; thus, the design effect is likely to exceed 1. This means that if effective sample size is considered, the match between achieved and effective may not be perfect.
5.1.2 Substitution Only Bulgaria, Cyprus, and Lithuania used substitution. Bulgaria did not provide reasons for substitutions. In Cyprus, after three unsuccessful attempts, a new unit was selected. In Lithuania, substitution was allowed for unsafe area, non-residential addresses, or if the addresses could not be accessed for any reason (buildings with security codes, uninhabited villages, etc). Substitution sample points were taken by continuing the random route from the previous visited apartment or house.
5.2 Sampling Errors Sampling errors result from the fact that not all units in the target population are surveyed. That is, instead of taking a census of the population, we take a sample; thus, a replication of the survey will result in a different sample and different estimates. Sampling error can be controlled through appropriate sampling design and sufficient sample size. Probability based samples can quantify the amount of sampling error present in the estimates—this is typically expressed as a confidence interval around the estimate.
Table 5.3 presents the estimates for six disability measures and the 95% confidence interval around these estimates.
(9) The loss in efficiency due to the use of a complex sampling design
(10
) Loss in efficiency due to complex survey design and weighting
Accuracy
21 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
5
Table 5.2: Minimum Required and Achieved Sample Size by Country
Country Minimum Required Sample Size Achieved Sample Size
AT 5,500 5,522
BE 5,800 5,804
BG 6,586 6,741
CY 3,000 3,002
CZ 5,800 5,807
DK 5,784 5,934
EE 4,253 4,253
FI 4,600 4,615
FR 15,727 15,768
DE 18,452 18,512
EL 7,564 7,567
HU 7,228 7,745
IS 3,000 3,007
IT 12,000 12,041
LV 4,675 7,515
LT 5,096 6,038
LU 3,000 3,004
MT 3,795 3,815
NL 7,000 7,011
NO 4,500 4,511
PL 12,713 12,938
PT 6,000 6,397
RO 9,854 9,962 (a)
SK 5,816 5,822
SI 4,561 4,641
ES 13,835 14,614
SE 5,500 5,500
UK 15,679 16,469
(a) Includes partial interviews
5 Accuracy
22 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
Table 5.3: Proportions, Standard Errors and Confidence Intervals for Six Key Disability Measures by Country
Country
Longstanding Illness or Health Problem
Severely Limited in Activities due to Health Problems for at Least the Past 6 Months
Prevented Buildings Use due to a Longstanding Health
Condition, Illness or Disease
Prevented Buildings Use due to Longstanding Difficulties
with Basic Activities
Prevented Paid Work due to a Longstanding Health
Condition, Illness or Disease
Prevented Paid Work due to Longstanding Difficulties
with Basic Activities
Propor-tion SE 95% CI
Propor-tion SE 95% CI
Propor-tion SE 95% CI
Propor-tion SE 95% CI
Propor-tion SE 95% CI
Propor-tion SE 95% CI
Austria 0.29 0.01 0.27–0.31 0.09 0.01 0.08–0.10 0.04 0.00 0.03–0.05 0.03 0.00 0.02–0.03 0.05 0.01 0.05–0.07 0.03 0.00 0.03–0.04
Belgium 0.26 0.01 0.25–0.27 0.07 0.01 0.06–0.07 0.04 0.00 0.04–0.05 0.03 0.00 0.03–0.03 0.07 0.00 0.06–0.08 0.04 0.00 0.03–0.04
Bulgaria 0.40 0.74 Not provided 0.06 0.29 Not provided 0.07 0.30 Not provided 0.05 0.28 Not provided
0.06 0.27 Not provided 0.02 0.17 Not provided
Cyprus 0.35 0.01 0.33–0.36 0.05 0.00 0.04–0.06 0.04 0.00 0.03–0.04 0.03 0.00 0.03–0.04 0.05 0.01 0.04–0.06 0.02 0.00 0.01–0.03
Czech Republic
0.26 0.01 0.25–0.27 0.05 0.00 0.04–0.05 0.05 0.00 0.04–0.05 0.02 0.00 0.02–0.03 0.05 0.00 0.05–0.06 0.01 0.00 0.01–0.01
Denmark 0.31 0.01 0.30–0.32 0.07 0.00 0.06–0.08 0.06 0.00 0.05–0.06 0.04 0.00 0.04–0.05 0.09 0.00 0.08–0.10 0.06 0.00 0.05–0.07
Estonia 0.43 0.01 0.41–0.44 0.09 0.00 0.08–0.10 0.05 0.00 0.05–0.06 0.02 0.00 0.01–0.02 0.05 0.00 0.04–0.05 0.01 0.00 0.01–0.01
Finland 0.70 0.01 0.69–0.72 0.06 0.00 0.05–0.07 0.03 0.00 0.03–0.04 0.03 0.00 0.03–0.04 0.02 0.00 0.02–0.03 0.02 0.00 0.01–0.02
France 0.32 0.01 0.31 –0.33 0.07 0.02 0.06–0.09 0.03 0.02 0.02–0.05 0.03 0.02 0.01–0.04 0.04 0.02 0.03–0.06 0.02 0.02 0a–0.03
Germany 0.41 0.01 0.40–0.43 0.12 0.00 0.11–0.13 0.06 0.00 0.05–0.07 0.04 0.00 0.04–0.05 0.08 0.00 0.07–0.08 0.03 0.00 0.03–0.03
Greece 0.34 0.01 0.33–0.35 0.07 0.00 0.06–0.07 0.06 0.00 0.06–0.07 0.02 0.00 0.02–0.03 0.03 0.00 0.03–0.04 0.01 0.00 0.01–0.01
Hungary 0.54 0.00 0.53–0.55 0.12 0.00 0.11–0.12 0.10 0.00 0.09–0.10 0.08 0.00 0.08–0.09 0.10 0.00 0.09–0.11 0.06 0.00 0.05–0.06
Iceland 0.27 0.01 0.25–0.29 0.06 0.01 0.05–0.06 0.02 0.00 0.01–0.02 0.01 0.00 0.01–0.01 0.06 0.00 0.05–0.07 0.01 0.00 0.01–0.02
Italy 0.22 0.01 0.21–0.23 0.05 0.00 0.04–0.05 0.04 0.00 0.04–0.05 0.02 0.00 0.02–0.02 0.02 0.00 0.02–0.03 0.01 0.00 0.01–0.01
Latvia 0.45 0.01 0.43–0.46 0.11 0.00 0.10–0.12 0.05 0.00 0.04–0.06 0.02 0.00 0.02–0.03 0.08 0.00 0.07–0.09 0.03 0.00 0.03–0.04
Lithuania 0.41 0.01 0.40-0.42 0.11 0.01 0.10-0.12 0.07 0.00 0.06-0.07 0.03 0.00 0.03-0.04 0.09 0.00 0.08-0.10 0.02 0.00 0.02-0.02
Luxembourg 0.27 0.01 0.25–0.29 0.07 0.01 0.06–0.09 0.04 0.00 0.03–0.05 0.02 0.00 0.01–0.03 0.05 0.00 0.04–0.06 0.02 0.00 0.02–0.03
Malta 0.51 0.01 0.49–0.53 0.05 0.00 0.04–0.06 0.03 0.00 0.03–0.04 0.02 0.00 0.01–0.02 0.01 0.00 0.01–0.01 0.00 NA NA
Netherlands 0.31 0.01 0.30–0.32 0.07 0.01 0.06–0.07(b) 0.04 0.00 0.04–0.05(b) 0.02 0.00 0.02–0.02 0.08 0.00 0.08–0.09 0.03 0.00 0.02–0.03
Norway (11) 0.33 Not provided 0.14 Not provided 0.04 Not provided 0.03 Not provided 0.11 Not provided 0.06 Not provided
(11) Estimates were updated based on frequency tables provided. Standard errors or Confidence Interval information was not available
Accuracy 1
1
5
EU Comparative Quality and Technical Report: European Health and Social Integration Survey 23
Country
Longstanding Illness or Health Problem
Severely Limited in Activities due to Health Problems for at Least the Past 6 Months
Prevented Buildings Use due to a Longstanding Health
Condition, Illness or Disease
Prevented Buildings Use due to Longstanding Difficulties
with Basic Activities
Prevented Paid Work due to a Longstanding Health
Condition, Illness or Disease
Prevented Paid Work due to Longstanding Difficulties
with Basic Activities
Propor-tion SE 95% CI
Propor-tion SE 95% CI
Propor-tion SE 95% CI
Propor-tion SE 95% CI
Propor-tion SE 95% CI
Propor-tion SE 95% CI
Poland 0.37 0.01 0.35–0.38 0.10 0.00 0.09–0.10 0.06 0.00 0.05–0.06 0.03 0.00 0.03–0.04 0.07 0.00 0.06–0.08 0.03 0.00 0.02–0.03
Portugal 0.33 0.01 0.32–0.34 0.05 0.00 0.05–0.06 0.05 0.00 0.04–0.05 0.02 0.00 0.02–0.02 0.05 0.00 0.05–0.06 0.01 0.00 0.01–0.02
Romania 0.24 0.01 0.23–0.25 0.05 0.00 0.04–0.05 0.08 0.00 0.08–0.09 0.05 0.00 0.04–0.05 0.05 0.00 0.05–0.06 0.02 0.00 0.02–0.02
Table 5.3: Proportions, Standard Errors and Confidence Intervals for Six Key Disability Measures by Country (continued)
Country
Longstanding Illness or Health Problem
Severely Limited in Activities due to Health Problems for at Least the Past 6 Months
Prevented Buildings Access due to a Longstanding
Health Condition, Illness or Disease
Prevented Buildings Use due to Longstanding Difficulties
with Basic Activities
Prevented Paid Work due to a Longstanding Health
Condition, Illness or Disease
Prevented Paid Work due to Longstanding Difficulties
with Basic Activities
Propor-tion SE 95% CI
Propor-tion SE 95% CI
Propor-tion SE 95% CI
Propor-tion SE 95% CI
Propor-tion SE 95% CI
Propor-tion SE 95% CI
Slovak Republic
0.43 0.01 0.42–0.44 0.10 0.01 0.08–0.12 0.03 0.00 0.03–0.04 0.01 0.00 0.01–0.02 0.03 0.00 0.03–0.04 0.01 0.00 0–0.01
Slovenia 0.34 0.01 0.32–0.35 0.07 0.00 0.06–0.08 0.07 0.00 0.06–0.08 0.04 0.00 0.04–0.05 0.04 0.00 0.03–0.05 0.02 0.00 0.02–0.03
Spain 0.39 0.01 0.36–0.42 0.05 0.04 0 (a)–0.13 0.05 0.04 0 (a)–0.13 0.04 0.05 0 (a)–0.13 0.06 0.04 0 (a)–0.14 0.04 0.05 0 (a)–0.14
Sweden 0.32 0.01 0.31–0.33 0.09 0.01 0.08–0.10 0.02 0.00 0.02–0.03 0.01 0.00 0.01–0.01 0.06 0.00 0.06–0.07 0.01 0.00 0.01–0.02
United Kingdom
0.63 0.01 0.62–0.65 0.39 0.01 0.37–0.40 0.04 0.00 0.03–0.04 0.01 0.00 0.01–0.02 0.05 0.00 0.04–0.05 0.01 0.00 0.01–0.01
(a) The lower bound of the confidence interval was set to 0 as the countries provided negative lower bound. (b) Mismatch between the reported standard error and confidence intervals.
5 Accuracy
24 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
5.3 Nonsampling Errors 5.3.1 Coverage Error Failure to include all members from the target population in the sampling frame yields coverage error. There are four major types of coverage problems—missing elements, foreign elements, duplication, and clustering. Missing sampling elements on the frame lead to undercoverage, which often requires a supplemental frame or some type of linking procedure. Sampling units that are not part of the target population (i.e., foreign elements), but are included on the frame, lead to overcoverage. The best way to avoid overcoverage is to clean the sampling frame before the sample selection occurs; however, this procedure is often costly. Duplication occurs when the same sampling unit is represented more than once on the sampling frame; thus, some units have a higher probability of getting selected relative to others. Duplication problems are usually resolved via elimination of duplicates or weighting each unit by the inverse of its number of listing and using selection probabilities proportionate to that number. Finally, clustering occurs when more than one sampling elements appear together under the same listing. Depending on the frequency and size of clustering, possible solutions are to include in the sample all elements within the cluster, select at random one element from the cluster, or relist the elements.
One way to evaluate the extent to which the selected sample is representative of the target population is to compare the unweighted respondent estimates to known estimates for the target population (usually, demographic characteristics). Table B-1 annex B presents such estimates for household composition, age, gender, education, region and urbanization by country. For most countries, the respondent pool is largely unbalanced on these characteristics. Not surprisingly, after post-stratification, the weighted estimates match or come very close to the population percentages (12). Several countries are excluded from Table B-1 - Denmark, (13) Finland, Poland and Slovakia provided population and respondent totals and we do not have information on the level of item missingness for these variables to be able to calculate percentages.
Certain data are missing for some countries either because the information was not included in the quality or technical reports provided, or it is not available in these countries. Additionally, no data on level of urbanization was included for Sweden and Netherlands.
5.3.2 Nonresponse Response rates are often used as a measure of survey quality, based on the fact that the probability sampling inference requires 100% response rate. When the latter cannot be achieved, the inferential paradigm does not pertain and often models of the impact of nonresponse on data quality are needed. Given the absence of strong correlation between response rates and bias (Groves and Peytcheva, 2008), response rates should be viewed as only one of many data quality indicators. Traditionally, face-to-face surveys yield the highest response rates, followed by telephone survey, mail and web. Figure 5.1 presents the response rates (14) by country and mode, as reported in the quality reports by each country.
One caveat in the calculation of the presented response rates is that they do not account for cases with unknown eligibility. Thus, in telephone surveys for example, ring-no-answer cases are likely included in the denominator (eligible units), even though it is unknown whether these are business lines, or otherwise nonworking numbers. This possibility might explain the drastically low response rates for telephone surveys observed in Figure 5.1. Countries taking part in the EHSIS were required to report sample size, number of eligibles, final disposition codes and unit response rate (the ratio of completed interviews to
(12) Most countries employed household composition, age, gender, education, region and urbanization in their final weights.
(13) The subcategories for each demographic variable were also not labeled; thus, rendering the data unusable.
(14) Calculated as ratio of completed interviews to the number of eligible elements.
Accuracy
25 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
5
eligible sample elements). In addition, contractors were asked to report AAPOR RR1 and RR3 response rates, but barely half reported one or the other. When we further examined the countries with very low response rate (under 15%), we noted that the reported RR1 and RR3 were the same as the unit response rate for Austria and Germany, but surprisingly, even lower (by about 1%) for Belgium, Luxemburg, Netherlands and Sweden. Slovakia, Estonia, Norway and the United Kingdom did not provide the AAPOR response rates.
Figure 5.1: Response Rates by Mode and Country
This suggests that in addition to possibly having house effects in the assignment of final disposition codes (even their definition), contractors were not consistent in the way response rates were calculated. Thus, a cross-country response rate comparison should be done with caution. We recommend future rounds of the survey to require standardize final disposition codes and outline a procedure for assigning them, along with detailed instructions how the response rates of interest should be calculated.
In most countries, the main reason for nonreponse was refusal to take part in the interview. Table 5.4 presents the distribution of nonresponse outcomes by country. In only 10 countries (BE, DK, EE, FI, LV, LT, MT, PT, RO and SE), the main reason for nonresponse was failure to contact the selected sampled person. The extent to which refusals and noncontacts are systematically different from those interviewed in the EHSIS on disability-related measures may introduce bias in the estimates reported by country. Most countries used post-stratification to adjust for nonresponse (see Adjustment Error section).
0
10
20
30
40
50
60
70
80
90
100
HUROES LV MTBG PT DK CZ IT EL CY SI PL FR LT IS FI EE LU SK DENO NL AT BE SE UK
Face to Face
Telephone
Country
5 Accuracy
26 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
Table 5.4: Percent of Nonrespondents in Each Country by Reason for Nonresponse
Country Percent Noncontacts
Percent Refusals
Percent Inability to be Interviewed
Percent Rejected Interviews due
to Interview Quality (a)
Percent Other
Nonresponse Total
AT 40.5 55.1 1.4 0.1 2.9 100
BE 61.2 36.5 1.6 0.7 0.0 100
BG 39.5 59.6 0.8 0.0 0.0 100
CY 36.9 57.5 0.4 0.1 5.2 100
CZ 8.1 83.1 1.3 0.0 7.6 100
DK 50.6 37.6 2.2 0.1 9.5 100
EE 83.6 16.0 0.0 0.0 0.4 100
FI 64.0 36.0 0.0 0.0 0.0 100
FR 36.8 41.4 9.7 12.2 0.0 100
DE 25.1 66.1 2.2 0.8 5.8 100
EL 0.0 70.7 4.8 0.0 24.5 100
HU 0.0 97.8 2.2 0.0 0.0 100
IS 45.6 47.1 5.7 1.7 0.0 100
IT 1.1 90.3 2.7 2.5 3.4 100
LV 57.0 40.8 1.7 0.0 0.5 100
LT 58.0 41.4 0.0 0.1 0.4 100
LU 19.7 63.1 5.7 1.1 10.3 100
MT 50.5 49.5 n/a 0.0 0.0 100
NL 41.9 54.5 0.9 2.7 0.0 100
NO 33.1 64.0 0.8 1.1 1.0 100
PL 35.1 64.9 0.0 0.1 0.0 100
PT 50.9 40.4 1.6 5.6 1.5 100
RO 76.0 24.0 0.0 0.0 0.0 100
SK 37.0 56.5 0.9 5.6 0.0 100
SI 28.4 71.6 0.0 0.0 0.0 100
ES 43.8 49.0 7.2 0.0 0.0 100
SE 53.5 28.4 1.0 17.1 0.0 100
UK 23.7 26.8 0.0 0.9 48.6 100
(a) The tender specifies that any questionnaire containing more than 50% item nonresponse must be rejected.
All countries were asked to provide an average rate for item missingness, minimum and maximum values. The reported averages ranged from 0% to 15%; minimum values ranged from 0% to 0.1%, and maximum values ranged from 0% to 50% across countries. The highest percent item missingness across countries was for household income, which is common in surveys due to the sensitivity of the information requested. Table 5.6 presents the average, minimum and maximum item nonresponse by country. France
Accuracy
27 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
5
did not provide any information on the percent item missingness in their data. Poland and Slovakia reported the highest average percent item nonresponse (8.4% and 8.9% respectively), but the reports do not provide further detail what is driving these rates.
Table 5.5 should be interpreted with caution—we discovered, that countries differed greatly in the way they reported item missingness in two aspects: 1) some countries reported item nonresponse rates only for the substantive variables, excluding all demographics, and 2) some countries excluded “Refused” and “Don’t Know” responses from the item missing count.
Table 5.5: Average, Minimum and Maximum Item Nonresponse Rates by Country Country Average Percent Minimum Percent Maximum Percent
AT 0.3 0.0 28.4
BE 0.0 0.0 0.0
BG 0.5 0.0 12.1
CY 0.1 0.1 15.0
CZ 0.6 (a) 0.0 50.3
DK 0.0 0.0 0.0
FI 0.1 0.0 16.0
DE 0.2 0.0 10.3
EE 0.6 0.0 17.6
EL 0.9 0.1 9.2
HU 0.2 0.0 22.7
IS 0.0 0.0 0.0
IT 0.4 (b) 0.0 41.2
LV 0.3 0.1 16.3
LT 0.4 0.0 8.8
LU 0.2 0.0 8.5
MT (c) 0.5 0.0 0.5
NL 0.0 0.0 0.0
NO 1.0 0.0 12.4
PL 8.4 0.0 54.0
PT 1.6 0.0 45.8
RO 3.1 0.0 35.1
SK 8.9 0.0 29.7
SI 0.2 0.0 30.2
ES 2.0 0.0 10.7
SE 0.0 0.0 0.0
UK 0.0 0.0 0.0
(a) Based on substantive questions only. (b) Based on substantive questions only.
(c) Reported “less than 1%” for average and maximum percent.
5 Accuracy
28 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
5.3.2.1 Imputation Only five countries employed imputation methods to correct for item nonresponse. Finland, Hungary and Poland used multiple imputation; Latvia and Portugal used single imputation—interval regression and hot deck, respectively. Denmark imputed for only one variable (JOBSTAT), as it was not collected in the survey and the register data they used were incomplete.
5.3.3 Measurement Error Measurement error can lead to bias in survey estimates. Quantifying measurement error can be difficult and, therefore, more emphasis is placed on attempting to control for measurement error through the survey implementation process.
Four sources of measurement error are usually identified:
1. The method of data collection can affect responses to survey questions depending on whether the survey is interviewer-administered in-person, over the telephone, or self-administered by the respondent without the presence of an interviewer.
2. The questionnaire itself may affect responses due to the subject matter or poorly designed questions and question wording, including the order in which the questions are asked.
3. The effect of the interviewers themselves can also lead to error in the estimates from a survey if the interviewer does not read questions correctly or confuses the respondent.
4. The respondents themselves may not provide accurate or adequate answers to all survey items if they misunderstand the questioning, may not provide a truthful response or may not respond at all.
All of these areas of potential measurement error as they relate to the EHSIS are discussed and described in this section.
5.3.3.1 Data Collection Mode The survey was designed for face-to-face administration, but in some countries the survey was administered using the telephone or via a web-based application. The decision as to which mode to use was based on cost and the appropriateness of a particular mode in a given country. For example, in some countries such as Denmark, internet coverage is greater than in some other countries and respondents are typically more comfortable with internet use. In addition, a suitable sampling frame is available for a web-based design. This differs from Germany where, in general, respondents are more used to surveys being carried out over the telephone.
Only two-fifths of the countries taking part (11 of 28) used computer assisted or paper and pencil face-to-face interviewing exclusively, while in 12 countries the survey was administered over the telephone. In Latvia both telephone and face-to-face modes of data collection were used. In Denmark respondents younger than 65 completed the survey themselves online while those respondents aged 65 or older were interviewed using the telephone because those in the older age group were more likely to respond by phone than via the web. Spain also used a mixed mode approach combining telephone and web-based interviews (see Table 5.6).
The use of different modes of data collection can lead to mode effects, where the same questions asked in different modes produce different results. Dillman 2007 considers these distinctions between modes and suggests that the way information is communicated to respondents can influence the thought process involved in interpreting questions and response options. Furthermore, category order effects, which can cause variations in the selection process of response categories, can occur as a result of these differences. The most common of these are primacy and recency effects. The primacy effect is the tendency to choose answer categories towards the beginning of a list of options. This effect is more common in self-
Accuracy
29 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
5
administered surveys or surveys where respondents are asked to read from a list shown on a card. In contrast, the recency effect is the tendency for categories towards the end of a list to be selected. This effect is more common in telephone-administered surveys where response options are read out by interviewers. Answer categories towards the end of a list tend to be selected more frequently because these are the ones respondents are more likely to remember, although sometimes not reflecting the respondents’ true circumstance or opinion (Dillman, 2007; Krosnick and Alwin, 1987). For telephone administration in EHSIS contractors should have individually prompted each response category (Yes/No) to ensure that all response options were noted by the respondent rather than simply reading out the list in its entirety. It is not entirely clear which method was followed for each of the contractors using telephone administration. However, over the telephone, both methods can cause respondent fatigue. As mentioned earlier, direction and testing and adaption for CATI mode should be included for any future rounds of the survey.
Countries conducting the survey using CATI reported that reading long lists over the telephone was cognitively burdensome for respondents making it harder for them to process the information received, and may have impacted on data quality. Contractors commented that the survey questions seemed repetitive to respondents and they became tired or lost interest.
Table 5.6: Mode of Data Collection by Country Country Mode of Data Collection
AT CATI
BE CATI
BG PAPI
CY PAPI
CZ CAPI
DK CAWI / CATI
EE CATI
FI CAPI
FR CATI
DE CATI
EL CAPI
HU CAPI / PAPI
IS CATI
IT CAPI
LV (15) CAPI/ CATI
LT CAPI
LU CATI
MT PAPI
NL CATI
NO CATI
PL PAPI
PT CAPI
RO PAPI
SK CATI
(15) PAPI was used as an alternative to the CAPI mode where interviewers experienced technical problems with the CAPI interview or were
concerned about using their laptops for safety reasons. The PAPI questionnaires were subsequently entered into the CAPI program.
5 Accuracy
30 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
5.3.3.2 The Questionnaire Translation A source questionnaire and interviewer instructions were provided in English . A guide to the variables used in the questionnaire is provided in annex 1 of this report.
Each contractor was required to translate the questionnaire into their target languages in order to carry out the survey in their respective country. Table 5.7 shows which languages were used during the survey administration for each country.
Table 5.7: Languages Used to Conduct the Survey by Country
Country Language(s) Country Language(s)
AT German LV Latvian; Russian
BE French; Dutch LT Lithuanian
BG Bulgarian LU French; Luxembourgish; German
CY Greek MT Maltese; English
CZ Czech NL Dutch
DK Danish NO Norwegian
EE Estonian; Russian PL Polish
FI Finish; Swedish PT Portuguese
FR French RO Romanian
DE German SK Slovak; Hungarian
EL Greek SI Slovenian
HU Hungarian ES Spanish; regional official languages (Catalan, Valenciano, Euskera, and Gallego); English
IS Icelandic SE Swedish
IT Italian UK English
A standard three-step translation protocol was prescribed for use, by Eurostat, which contractors were required to follow. According to this protocol, each contractor had to:
Step 1: Translate the source questionnaire into the target language(s). The translator should work in the health/social statistics field, understand the health/disability. concepts used and use the target language as mother tongue and English as a working language.
Step 2: After the initial translation, a checker with the same characteristics as the original translator checks the translation while making use of the interviewer instructions to inform their conceptual understanding.
Step 3: The checker’s views and those of the initial translator are then bought together in a final translation. If they do not agree, a third expert is solicited to make the final decisions.
Countries sharing the same language could organise a coordinating translation process in order to obtain the same questionnaire or a version of it taking into account the linguistic and cultural differences. This occurred in the case of Austria. The Austrian questionnaire was based on the German questionnaire whilst making adaptions to include the Austrian demographic standards to the EU core social variables. In Slovakia the questionnaire was also translated into Hungarian. The Hungarian contractor provided the
Accuracy
31 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
5
translated questionnaire and minor adaptions were made to suit the Slovakian context.
Each country was required to complete a translation report form detailing where problems in translation occurred and their proposed solution. The forms were vetted as the survey development progressed and, where necessary, the country coordinator contacted. Only Greece and Lithuania appear to have deviated from the translation protocol, using a back translation process from English to their national language and then back to English.
No deviation from the original questionnaire was allowed except those resulting from cultural differences. The question wording, meaning and order in which the questions were asked had to remain consistent cross-nationally. Additional questions or modules could not be added to the questionnaire since it was considered that this would alter the contextual mindset of the respondent, add to the length of the interview and increase respondent burden.
It is difficult to know whether or not any major deviations from the original source questionnaire exist if not explicitly reported, since national translations were not checked. In general, only minor changes were reported when it was necessary to ensure that the questions were conceptually equivalent. For example, the introduction to the employment section was cited by Greece and France in this regard. Finland reported that one of the answer options to the barriers to education question (EdPrv) was ambiguous when translated directly into Finnish and, therefore, additional instructions were added to the existing interviewer instructions.
“Difficulties in getting to learning facility.” Some have wondered if this meant physical accessing to a learning facility, or difficulties being admitted to a school. This was somewhat ambiguous in Finnish translation. The former was the correct, this was explained in the interviewer instructions; thus, no corrections to the questionnaire were needed. “[FINLAND]
However, a translation error was identified in the German questionnaire following the completion of fieldwork that did change the conceptual meaning of the last of the MEHM (16) questions (MEHMH3). Respondents were asked to assess the severity of their longstanding health problem in terms of how it impacted on their ability to carry out basic activities for more than 6 months by asking: ‘for at least the past 6 months’. The German translation asked only about the period ‘over the past 6 months’. This translation error would also have affected the Austrian data collection because the correction was not made in the Austrian questionnaire until 1 month prior to the end of fieldwork.
Pretesting All of the countries conducting the survey carried out some form of pretesting prior to main stage field work in the form of a pilot study and interviewer debrief. Pretesting was carried out to ensure that the data collection instrument was programmed correctly and that supporting systems were correctly operationalized. Length of interview and respondent engagement was also monitored.
All but five countries (Austria, Germany, Luxembourg, Norway and Slovakia) also carried out pre-field testing in the form of cognitive interviewing. Since the previous version of the EHSIS questionnaire had been extensively cognitively tested and piloted (see section 2.1) the questions themselves were fixed and their meaning described in the interviewer instructions. Therefore, the purpose of any cognitive testing at this stage was to test and support the questionnaire translation to ensure the best translation for the underlying concepts. Hence, literal translations may have needed adaptation if they did not reflect the meaning of the original questions.
(16) Mini-European Health Module
5 Accuracy
32 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
Changes Made to Questionnaire Question sequence. All of the survey contractors followed the sequence of the questionnaire except for the United Kingdom—see table C-1 in annex C. In the United Kingdom respondents were asked for their postcode in order to derive the information required for the ‘region of residence’ and ‘degree of urbanisation’ core social variables. Respondents were also asked household grid questions in order to obtain the information needed for the ‘household composition’ core variable. The UK contractor felt that collecting this information near the start of the interview affected response rates because respondents terminated the interview at that point. As a result, the postcode question and most of the household grid questions (apart from the question asking for household size) were moved closer to the end of the questionnaire, between the ‘country of birth of the mother’ question and the ‘net monthly income of household’ question.
Question modifications. Modifications made to the questionnaire and a summary of questions that proved problematic during field work are described in the contractors own words in table C-1 annex C.
In general, minor adaptions were made to the question wording to account for the mode of administration. As Austria pointed out, in a CATI survey where telephone numbers are generated, it is not possible to send out an advance letter; therefore, the interviewer introductions were adapted accordingly. Another example of the kind of adaptions that were made because of mode, relates to the show card instruction “Please use this card as a guide and choose all that apply,” which could be changed to “I am going to read out some reasons, please tell me those that apply to you.” Furthermore, in the barrier questions, if the survey is CATI, each response option should have been individually prompted in order to try and provide a common stimulus across modes and avoid potential mode effects such as primacy and recency effects see section 3.3.3.1. Where the response options were individually prompted some contractors chose not to read out the last category ‘No, nothing prevents me from .....’ since if the respondent had not answered ‘yes’ to any of the barriers during the individual prompt, including the ‘other category’, then it could be assumed that the option ‘No, nothing prevents me…’ applied.
In general, adaptions made were designed to minimize mode effect and maintain comparability cross-nationally. However, some of the adaptions may have caused variations. Denmark, for example, appeared to have added a filter question in the employment section asking ‘Do you want a paid job?’. It is unclear how this was administered and if respondents had the opportunity to hear all of the barrier reasons that might apply to them before answering. The Slovenian contractor added a category to the list of barriers provided to the question about access to private vehicles ‘I do not have anyone to take me’. In this regard, the Slovenia data may not be comparable to other countries.
With regard to the EU core social variables, the Spanish contractor indicated that they simplified the questioning relating to household composition, but it is not clear how the data would be affected by this modification. The Norwegian contractor did not collect Net Monthly Income according to the specifications of the EU Core Social variables, but instead collected Gross Annual Income. The income data from Norway will not be comparable with other countries. Italy reported a reluctance by respondents to report income at all and suggested a shorter list of income bands. Italy also reported that because it was no longer compulsory to report marital status on official documents such as identity cards, respondents were more reluctant to provide marital status information in surveys.
Interview duration. The EHSIS interview was expected to take around 40 minutes in total when administered by an interviewer in a face-to-face mode. Across all countries the average time taken to conduct the interview was just over 25 minutes. However, it is worth noting that some of the minimum duration times reported were extremely low and were qualified with explanations about technical issues in reporting the calculations. This may have bought down the average duration time.
Table 5.8 shows that Estonia and Iceland had the lowest averages (14 minutes and 15 minutes, respectively), while Cyprus and Lithuania had the highest averages (42 and 40 minutes, respectively). The longest interviews took place in Hungary where a maximum length of 3 hours and 48 minutes was reported, and Spain where a maximum length of 3 hours and 15 minutes was reported. Some further investigation as to why some interviews took so long would be useful to inform any future rounds of the survey.
Accuracy
33 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
5
The face-to-face surveys took just over 7 minutes longer to administer on average than the telephone and web surveys.
Table 5.8: Interview Duration by Mode of Administration
Country Mode Minimum Length (mins)
Maximum Length (mins)
Average Length (mins)
AT CATI 10
105 22
BE CATI 1 88 20
BG PAPI 15 100 34
CY PAPI 20 60 42
CZ CAPI 15 135 28.16
DK CATI CAWI
5 5.7
94.2 90.5
15.45 16.62
EE CATI 5 50 14.2
FI CAPI 4 145 17
FR CATI 13 164 29
DE CATI 10 145 29
EL CAPI 30 65 33
HU CAPI/PAPI 7 228 22
IS CATI 2 67 15
IT CAPI 14 90 19.5
LV PAPI/CAPI/ CATI
2 1
76 68
17.6 16.4
LT CAPI 29 60 40
LU CATI 15 96 32
MT PAPI 20 85 38
NL CAPI 0 73 23
NO CATI 8.33 79.10 20.4
PL PAPI 16 90 38
PT CAPI 9.41 169.4 23.59
RO PAPI 10 150 36.47
SK CATI 4.32 118 16.33
SI CAPI 10 150 22
ES CATI/ CAWI
12
Not available
195
Not available
32.14
Not available
SE CATI 0 76 20
UK CATI 8 129 25
5 Accuracy
34 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
5.3.3.3 The Interviewer The role of the interviewer is important in household surveys that necessitate face-to-face data collection or those that require interaction with a respondent in order to obtain their participation, such as phone surveys. Often acting as the sole representative of a survey organization, interviewers are tasked with gaining a participant’s trust, accurately recording their responses, and transmitting those data securely.
Interviewer effects have been well-studied and can impact the quality of the data collected. Several quality indicators were reported to ensure high performance during fieldwork. Quality assurances related to interviewer recruitment and training included the duration of training, level of interviewer experience, and type of training administered. Other quality indicators included interviewer education level and prior interviewing experience, the ratio of interviewers to supervisors, the ratio of respondents to interviewers, and the mean number of contact attempts made by interviewers before marking a case nonparticipation.
Interviewer Recruitment One requirement for this survey was that only interviewers who had finished lower secondary education (ISCED 2) and those with previous experience in population surveys should be used during fieldwork. A majority of contractors recruited interviewers with at least an education level of ISCED 3-4 or higher.
In addition to education, countries were asked to provide details on the age range and economic activity status of their interviewers. A majority of contractors used interviewers that primarily fell into two age groups: 15-29 (reported by 10 countries) and 45-59 (reported by 9). Six contractors reported that the highest percent of their interviewers were between the ages of 30 and 44. Interviewers recruited also tended to be primarily employed, reported by 15 countries. Five countries reported using a majority of students as their interviewers. The majority of contractors used experienced interviewers employed directly by the organization that fielded the survey and those with experience working on social surveys.
Interviewer Training Training interviewers and their supervisors is integral to ensuring that an instrument is administered correctly, high participation rates are achieved, and data are collected accurately. Effective training can minimize interviewer effects and should provide a background to the project and include training in the mode(s) of data collection. Good training should also allow time for practice interviewing and discussions on how to handle difficult cases (e.g., refusals).
Specific survey-related training sessions were required for both the interviewers and their supervising staff. The type of training varied; however, all contractors reported providing training to their interviewers prior to data collection. Finland conducted interviewer training via telephone, while Poland used e-training. Other contractors reported training sessions that incorporated role play and practice sessions. Some countries that used CAPI data collection also trained interviewers on computer-assisted data entry and other CAPI exercises. Other methods of interviewer training included monitoring the first live interviews and holding follow-up debriefings to discuss areas where more guidance was needed. Other contractors required their interviewers to perform test interviews before being permitted to start fieldwork.
It was recommended that a 15% subsample of interviews be back checked to ensure data quality. Other quality checks implemented on the EHSIS included monitoring a random sample of live and previously recorded interviews, cross-checks of all questionnaires submitted, daily performance and progress monitoring, including collecting refusal and response rates, call times, and number of attempts. Re-interviews were also conducted. Ongoing feedback was given and additional training was provided as needed.
Length of interviewer training varied from 2 hours (Belgium, Czech Republic, Iceland, Netherlands, and Sweden) to 3 days (Spain). Poland’s interviewers had access to online training materials for up to 2 weeks. The average amount of time spent on interviewer training was a little over half a day, or 5.6
Accuracy
35 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
5
hours (17)
Interviewer/Supervisor Ratio The ratio of interviewers to supervisors provides a good measure of interviewer performance as it may be indicative of the level of quality monitoring by each organization.
There is likely little difference between interviewer/supervisor ratios observed in face-to-face surveys (PAPI or CAPI); however, one would expect a higher ratio on telephone surveys where a single supervisor could monitor a large group of interviewers located in a single location. It is more difficult to supervise the same number of interviewers in a field setting and when daily travel is required. Therefore, we might expect to see a lower ratio of interviewers to supervisors in CAPI and PAPI-administered surveys.
The country that reported the highest interviewer/supervisor ratio was Italy with 139 interviewers to every 1 supervisor and collected data via CAPI. The lowest ratio of interviewers to supervisors was reported by Denmark, approximately 4:1. Denmark mainly utilized CATI, but also used a web mode for respondents older than 65. The average interviewer to supervisor ratio for telephone surveys was 22.6, and 27.9 (18) for face-to-face interviews. It is unknown why these data trend in the opposite direction as would be expected. We can only hypothesize that they are the result of varying national standards including differences in supervisor titles or the organizational structure of survey operations and possibly interviewer availability.
Respondent/Interviewer Ratio Mode of data collection is likely to affect the ratio of respondents to interviewers. One might expect CATI survey interviewers to be able to survey more participants than modes that require face-to-face data collection such as PAPI and CAPI. This trend is reflected in the average ratios reported. The average ratio for CATI interviews was 60 respondents to every interviewer; the average ratio for the PAPI and CAPI modes was 46 respondents for every interviewer.
Table 5.9 displays the respondent/interviewer ratio as the number of respondents to one interviewer by mode of administration.
(17) Poland provided e-training materials for up to 2 weeks and therefore are excluded from this mean analysis.
(18) Countries that collected data in multiple modes and only reported one ratio are not included in this analysis: Denmark, Latvia, and Spain.
5 Accuracy
36 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
Table 5.9: Respondent/Interviewer Ratio by Country and Mode of Administration
Country Mode Ratio
BG Face-to-Face 33.5
CY Face- to-Face 83
CZ Face-to-Face 17.44
FI Face-to-Face 71
EL Face-to-Face 68
HU Face-to-Face 20.4
IT Face-to-Face 43.5
LT Face-to-Face 67
LV Face-to-Face and telephone 117
MT Face-to-Face 54.5
PL Face-to-Face 31.3
PT Face-to-Face 67.3
RO Face-to-Face 12.77
SI Face-to-Face 27
AT Telephone 19
BE Telephone 72
DK Telephone 76.8
EE Telephone 70
FR Telephone 148
DE Telephone 32
IS Telephone 97
LU Telephone 19
NL Telephone 50
NO Telephone 46
SK Telephone 44
ES Telephone 365.5
SE Telephone 95
UK Telephone 31
Spain reported the highest respondent/interviewer ratio, where data were collected via CATI and CAWI, and reported 365.5 respondents to every interviewer. The lowest ratio was observed in Romania, who utilized PAPI data collection and reported 13 respondents to every interviewer.
The survey required that the respondent/interviewer ratio should not exceed 70, as EHSIS was designed for face-to-face data collection. In seven countries this limit was exceeded. Two countries (Cyprus and
Accuracy
37 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
5
Finland) administered the survey face-to-face; while the remaining five (Belgium, Denmark, France, Iceland and Sweden) administered the EHSIS via telephone.(19)
Contact attempts made before declaring nonparticipation. It is equally important to encourage selected respondents to take part in the survey. As such, the mean number of contact attempts before declaring nonparticipation was also reported. However, it should be noted that it is difficult to make direct comparisons cross-nationally as respondent behaviours and the ease with which contact can be made will vary by country and survey mode.
In general, countries administering the survey face-to-face reported lower mean number of contact attempts before declaring non-participation than did those via telephone. This is common due to higher costs of in-person attempts and their greater effectiveness of resulting in an interview. Table 5.10 displays the mean number of contact attempts before declaring non-participation by mode of administration.
Table 5.10: Mean Number of Contact Attempts by Country and Mode of Administration
Country Mode Mean Number of Contact Attempts
BG Face-to-Face 2.14
CY Face-to-Face 3
CZ Face-to-Face 1.4
FI Face-to-Face 1.8
IT Face-to-Face 1.4
LT Face-to-Face 1
MT Face-to-Face 3
PL Face-to-Face 1.68
PT Face-to-Face 3
RO Face-to-Face 3.5
SI Face-to-Face 3
BE Telephone 8
DK Telephone 6.9
EE Telephone 1.3
FR Telephone 18
NO Telephone 5.4
SK Telephone 4.94
ES Telephone 24.2
UK Telephone 6.1
The average number of contact attempts made in the face-to-face surveys was 2.3. The average number of contact attempts was not recorded for Greece, Hungary or Latvia. The average number of contact attempts made for the telephone surveys was 9. Six countries that administered EHSIS by telephone did not report the average number of contact attempts.(20) Denmark and Spain used a mixed-mode approach to data collection (phone and web) and are included in the analysis of mean contact attempts for telephone.
(19) Latvia reported 117 respondents to every interviewer, but only provided a single ratio for both modes of data collection.
(20) AT, DE, IS, LU, NL, SE.
5 Accuracy
38 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
Spain reported the highest mean number of contact attempts at 24.2. The lowest average number of contact attempts was reported by the Lithuanian contractor at 1. This survey was administered face-to-face and collected data for approximately 4 months.
5.3.3.4 The Respondent Partials Partial interviews are interviews where the whole survey is not completed due to break off or other reasons. The survey required that partials with less than 50% responses were not counted as interviews. Only one country, Italy, reported partials, but it is unclear if they were included in the total number of interviews, as was the case with Romania.
Proxy Interviews Proxy interviews typically induce measurement error in the survey estimates as proxies are asked to report on someone other than themselves, for whom they may not have the most accurate information. This is especially true for private behaviours, subjective perceptions, or sensitive topics. Most countries allowed for proxy interviews, but employed stringent rules as to when proxies should be allowed. Since many of the questions in the EHSIS interview are subjective and rely on respondent’s own assessment of their situation, proxy interviews were permissible only when the sampled person was severely impaired. Table 5.11 presents the percentage of proxy interviews out of all interviews by country, among the countries that allowed proxy responding. Four countries (Cyprus, Hungary, Poland and Spain) reported higher than 5% proxy interviews. It appears that this was because these countries allowed a broader definition of a proxy interview, for example allowing proxy interviews when the selected sample person was absent from the household for education purposes or absent from the household for ‘other’ reasons.
Table 5.11: Percent Proxy Interviews by Country
Country Percent Proxy Interviews
BE 0.5
BG 1.0
CY 7
CZ 1.4
FI 0
FR 2
EL 1.5
HU 14.3
IS 0.1
IT 5.0
LV 1.7
LT 1.0
MT 3.5
Accuracy
39 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
5
Table 5.11: Percent Proxy Interviews by Country (continued)
Country Percent Proxy Interviews
NL 0.6
NO 0.4
PL 6.3
PT 3.8
RO 4.0
SK 1.2
ES 11.1
SE 0.1
UK 0.2
5.3.4 Data Processing Error Another stage of the survey process where error can be introduced is the data processing stage. The data processing stage comprises data entry, coding and data cleaning, and weighting adjustments.
5.3.4.1 Data Entry, Coding and Cleaning Errors may be introduced at this stage when data are miss keyed by the interviewer or the data entry specialist or coding is not accurate and coder bias may be apparent, range and consistency checks are not in place, computations are incorrect and/or data are not checked and validated thoroughly. This section of the report describes the data entry, coding and cleaning process as it was carried out cross-nationally and attempts to identify any issues that could affect the final data outputs where they are described by the contractors.
It should be noted that the mode of data collection will affect the way in which the data processing is carried out. For countries using computer-assisted interviewing (CAI) methods, routing, range and consistency checks were integrated within the computer program and carried out automatically, to a greater or lesser extent, depending on whether the survey was interviewer-administered (CAPI/CATI) or self-administered (CAWI). This limited the amount of data processing and editing needed at the end of fieldwork. Derived variables were computed within the interviewing programme or following data collection. For example, the number of persons in the household could be computed from the household grid information; the region and degree of urbanization from the sample frame information. Coding of open-ended questions was carried out after data collection was completed.
“Interviews conducted using CAPI…so most of the validation rules were already implemented in the script used to collect the data. Open questions were coded by two people with experience and then 100% of the coding was revised.” [Portugal]
The data processing stage can also be used to monitor the outputs and improve data quality as fieldwork progresses. Some countries stated that they were able to minimize item nonresponse for example, by identifying where proportions were high and retraining or replacing interviewers reporting a high percentage of items missing.
As shown earlier in table 5.6, five countries used PAPI administration, namely: Bulgaria; Cyprus; Malta; Poland; Romania. When PAPI administration was used, a separate data editing and coding programme was required. These countries provided specific EHSIS training for their coders and editors. It should also be noted that quality checks on coding and editing were also carried out by the supervisory staff. This is important for reducing errors and also avoiding coder bias. The countries following this process checked back directly with respondents for clarification where errors were found.
5 Accuracy
40 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
“The data processing … specialist responsible for EHSIS project made additional cross-check over the entered data. Each appeared mistake was first checked in the hard copy questionnaire. If the problem was not due to the entry activity but due to interviewing, a phone call to the respective respondent was made to clarify the information.” [Bulgaria]
“Data was entered using a data entry program which was specifically designed for this survey. This program had a number of in-built validations which helped minimise biases during the data entry stage. Data checking/editing was conducted in three main phases: i) Physical checking of the questionnaire when delivered to the office; ii) Checking during data entry; iii) Data was then filtered during the data analysis stage using numerous validations and logical tests.” [Malta]
“During the data entry process a screen level validation process and control and editing of errors was done by the data entry operators. There was a range of values checked for each variable. About 20% of the entered questionnaires were checked randomly for any other data entry errors (possible errors that did not conflict with the value range checks implemented in the IT application).” [Romania]
All contractors were asked to report where errors were detected and what had been done to minimize these errors. In general this process did not appear too problematic, although some reported that it was time-consuming. The errors that were detected were mainly resolved. Table 5.12 presents comments made by contractors in this regard. These quotes were selected because they were considered important for reference in future rounds of this survey.
Accuracy
41 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
5
Table 5.12: Data Processing Errors/Issues Detected by Country and Mode of Administration
Country Mode Main Errors Detected / Issues
AT/DE/LU CATI ▪ One consistency check was missing which could have been installed in the course of programming, that was comparison between the number of persons in the household (HH-Anz) and the number of persons taken up in the household matrix question (HHMX, HHMXage). For some other generated variables referring to the number of household members, the household matrix data was taken as reference. Only if the household matrix data was incomplete, it was referred to the information out of the variables HH_Anz & HH_Alter.
FI CAPI ▪ There were 141 cases where the respondent either didn’t know the number of persons in the household, their employment status, or the answer was some other way inconsistent. These may have been also punching errors by the interviewer.
We were able to eliminate these inconsistencies; we didn’t have to eliminate answers from the database. We were able to determine from the questions the right answer.
EL CAPI ▪ MARSTDEFACTO: incorrect coding in some cases
▪ JOBISCO-HATLEVEL: inconsistencies between the two variables
▪ HHINCOME: unreasonable and unusual numbers
▪ HHTYPE: wrong coding of “other household”
All variables were checked for errors and whenever necessary call backs have been made.
HU CAPI / PAPI
▪ Income data entry was not correct in 175 cases.
▪ Refusal codes were not correctly entered in 56 cases.
MT PAPI ▪ The largest share of errors was found in the core variables module (mainly discrepancies between household type and household size / number of workers).
▪ In some cases, inconsistencies were noted between different modules of the questionnaire (e.g., general health good or very good; however suffers from many health problems / disabilities)
In these cases, the respondents were re-contacted in order to verify these inconsistencies
PL PAPI ▪ Punching errors—correction in database.
▪ Questions that should be omitted because of filters—answers deleted from database.
▪ We also discovered 77 cases when respondent didn’t know the age of household member. In 71 cases we were able to determine age range, in 6 questionnaires it was impossible and they were deleted from the database.
RO PAPI ▪ There were errors regarding the date of the interview (wrong year or month entered).
▪ Also the start time and finish time (the interview started at a later time than the when it ended-the start time and end time were reversed when entering the data). There were about 30 such cases.
▪ The relationships between the members of the household (regarding the household questionnaire) were in some cases (about 40-50 cases) incorrectly entered.
5 Accuracy
42 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
Table 5.12: Data Processing Errors/Issues Detected by Country and Mode of Administration (continued)
Country Mode Main Errors Detected / Issues
SK CATI ▪ Because of the programmed questionnaire there were minimum mistakes. Due to the missing control in the questionnaire (HHLABOUR_EMP). We had to code “-1” because some respondents did not mentioned (count) some family members. It could be influenced by the fact that they did not count students or mothers on maternity leave.
UK CATI ▪ For 8 interviews, the interview length was not recorded accurately and so interview lengths for them are not available.
▪ In 18 instances, invalid dates of birth were recorded for the respondent. In 26 cases this also happened for other household members. In such instances, the CATI script calculated age from the year of birth alone and this age has been used in the data.
▪ 17 respondents were not routed to the questions used to derive JOBISCO and LOCNACE.
▪ 1 respondent did not code 11 or 12 at the question used to derive MAINSTAT but was routed to, and gave responses at, the questions used to derive JOBSTAT, JOBISCO and LOCNACE.
▪ 1 respondent was coded as null at all of the sub-variables for LHCD and LDBA.
5.3.4.2 Weighting Adjustments Almost all countries created selection weights and post-stratification weights. Twenty countries (21) reported post-stratifying their data by various auxiliary variables (mostly demographic) to correct for coverage and nonresponse error. Table 5.13 describes briefly the weighting procedure employed by each country.
21
It was unclear from the reports of the rest whether post-stratification was employed or not.
Accuracy
43 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
5
Table 5.13: Weighting Procedure by Country
Country Weighting Procedure
AT Probability weight applied (Horvitz-Thompson estimator) to account for the dual-frame approach; calibration using generalized regression estimator input (starting weight for the calibration is the probability weight). Auxiliary variables: joined distribution of age group x, sex, highest level of education, employment status, degree of urbanization, household size, federal state.
BE For the landline sample, within household probabilities of selection were calculated (the selection probability weight for the mobile sample was set to 1). The overall sample was then weighted in order to reflect the national population by region, gender, age, and highest level of education.
BG A post-stratification weighting procedure by sex, age and size of residence place, considering the latest available data on Bulgarian population. (The probabilities of selection for each housing unit were unknown due to the random walk approach).
CY Two sets of weights —household level probability of selection (inverse of the number of eligible persons in the household) with a maximum size of 4, and post-stratification. The probabilities of selection for each housing unit were unknown due to the random walk approach. The post-stratification weighting was carried out using Raking Adjustment for Nonresponse (the selection probability weights were used as base). Weights were developed by making the marginal distributions of the auxiliary variables (Age by Sex combination and degree of urbanisation) in the sample conform to the population marginals.
CZ A respondent’s design weight was calculated as the inverse of the product of three inclusion probabilities—the inclusion probability of a specific municipality (stage 1); the inclusion probability of a specific household in that municipality (stage 2) and the inclusion probability of one adult/respondent in this household (stage 3). At the end, the design weight was rescaled by number of 15+ population, by strata, in order that the sum of the design weight in all 32 stratum could be equal to the total 15+ population number from the Czech Republic.
A post-stratification weighting was carried out, in order to adjust the sample to selected control
Totals. The raking procedure included age by NUTS2, activity by NUTS2, urbanization, and highest level of education. The final weights were trimmed (a maximum for the weights was set at 3.0).
DK Clan-tool. Following variables used: Ancestry, education and family type. Sex, age and income have been part of the weighting, each by itself.
EE Each respondent unit was assigned a sampling weight to make the respondent’s contribution larger or smaller to compensate for the fact that persons with certain characteristics are not as reachable and likely to respond to the study, and for the data to be representative of the Estonian population. The census data of Statistics Estonia on distribution of the universe according to the set of demographic characteristics was used for weighting: gender, age, settlement type and main language according to five regions (NUTS 3).
FI Inclusion probabilities where corrected in order to match marginal distributions of several demographic variables like age, gender, marital status, educational level. Correction was performed by means of quadratic programming with objective function proportional to squared sum of individual sampling weight change.
FR Data weighted by sex, age, region, HHNPERS, education and DEGUR using Rim weighting. Weights were trimmed at 3.
(continued)
5 Accuracy
44 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
Table 5.13: Weighting Procedure by Country (continued)
Country Weighting Procedure
DE Weighting Step 1: Design weight (Probability weight), based on the Horvitz-Thompson estimator (inverse of the sampling probability) accounting for the dual-frame approach. Weighting Step 2: Calibration (GREG estimator) to account for selective Nonresponse. Auxiliary Variables: joined distribution of age group x sex, highest level of education, employment status, degree of urbanization, household size, Federal, state.
EL Two sets of weights —household level probability of selection (inverse of the number of eligible persons in the household) with a maximum size of 4, and post-stratification. The probabilities of selection for each housing unit were unknown due to the random walk approach. Post-stratification weighting used a combination of sex and age with 8 levels, region (NUTS 2 level) and degree of urbanization; the last two were also included in the sample stratification.
HU Design weight is the reciprocal of the inclusion probability for each selected individual. For respondents the design weights were multiplied with a nonresponse adjustment factor at locality by age group level, where the factor is the ratio of the number of selected individuals in scope and the number of respondents. Calibration was done at regional level. In each region we had 48 population totals the sample was to match with. Population by gender by age group, population by household size, population size group-of-locality by gender by broader age group.
IS Probability of selection weights were not calculated as the sampling frame was a population register at the individual level. The data were weighted by region, gender, age, and education.
IT A respondent’s design weight was calculated as the inverse of the product of three inclusion probabilities—the inclusion probability of a specific municipality (stage 1); the inclusion probability of a specific household in that municipality (stage 2) and the inclusion probability of one adult/respondent in this household (stage 3). At the end, the design weight was rescaled by number of 15+ population, by strata, in order that the sum of the design weight in all 48 strata could equal to the total 15+ population in Italy.
A post-stratification weighting was carried out, in order to adjust the sample to selected control totals.
The raking procedure included NUTS2, urbanization, age by gender, activity and highest level of education. The final weights were trimmed (a maximum for the weights was set at 3.0).
LV The design weights were computed as inverse of the sample inclusion probabilities. The design weights were adjusted by constant in each stratum to match the population size of each stratum. The adjusted weights were calibrated to the known population totals by the linear calibration using demographic and disability data (from population registers).
LT Post-stratification weighting was carried out using Raking Adjustment for Nonresponse (raking). Weights were developed by making the marginal distributions of the auxiliary variables (Age x Sex combination and degree of urbanisation) in the sample conform to the population marginals. Selection probability weights were incorporated by using them as starting weights in the procedure.
LU Calibration using generalized regression estimator (GREG). Input (starting weight) for the calibration is the design weight (probability weight). Auxiliary variables: joined distribution of age group x, sex, highest level of education, employment status, degree of urbanization, household size
MT Post-stratification weighting, basing on age, sex and district (LAU 1)
(continued)
Accuracy
45 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
5
Table 5.13: Weighting Procedure by Country (continued)
Country Weighting Procedure
NL For the landline sample, within household probabilities of selection were calculated (the selection probability weight for the mobile sample was set to 1). The overall sample was then weighted in order to reflect the national population by region, gender, age, and highest level of education.
NO The final weights for the survey have been calculated as the product of nonresponse weights and design weights. The basis for the calculation of nonresponse weights is population data given by Statistics Norway in 2013. The population data splits the Norwegian population into subgroups based on gender (2 subgroups), geographic region (7 subgroups), age (6 subgroups) and education (2 subgroups). A rim weighing algorithm is applied to the survey data, giving the nonresponse weights.
PL Data weighted by marital status, gender, voivodship, urbanization density and education.
PT Two sets of weights were used —design and calibration weights. The design weights accounted for probabilities of selection at the PSU, housing unit and household level. The calibration weight post-stratified the data by age, gender, marital status, degree of urbanization, education, number of people in the household, working status.
RO Probability of selection weights, adjustment for nonresponse by county and urbanicity (response homogenous group method), and post-stratification by age, gender, area of residence and region.
SK Post-stratification using Iterative Proportional Fitting procedure to adjust weights in order to fit population marginal distributions of age, gender, education, region.
SI Survey weights were obtained with an automatic iterative weighting (“‘raking”)’ procedure, using population margin for sex, age groups, NUTS-3 regions, highest level of education completed and degree of urbanisation.
ES Calibration by age, sex, urbanicity, nationality, and household size.
SE For the landline sample, within household probabilities of selection were calculated (the selection probability weight for the mobile sample was set to 1). The overall sample was then weighted in order to reflect the national population by region, gender, age, and highest level of education.
UK The landline sample was weighted for noncontact, refusal and selection of the individual; the mobile sample was weighted by number of mobile phones. Final weights were a combination of mobile and landline weights, correcting for age, gender and telephone status.
6 Preferences
46 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
6. REFERENCES Agafitei, L. (2012). Recent developments on disability statistics in the European Union. Presented at the 10th meeting of the Washington Group on Disability Statistics http://www.cdc.gov/nchs/ppt/citygroup/meeting10/WG10_Session1_3_Agafitei.pdf
Biemer, P. Groves, R. M., Lyberg, L. E., Mathiowetz, N. A. and Sudman, S. (1991). Measurement errors in surveys. New York: Wiley & Sons.
Biemer, P. (2010). Total survey error: Design, implementation, and evaluation. Public Opinion Quarterly, 74 (5): 817-848.
Biemer, P. (2010). Total survey error. In M. Lovric (Ed.), International encyclopedia of statistical science. New York: Springer.
Biemer, P. and Lyberg, L. (2003). Introduction to survey quality. New York: Wiley & Sons.
De Leeuw, E. D., Hox, J. J., and Dillman D. A. (2008). International handbook of survey methodology. New Jersey: Lawrence Erlbaum Associates.
Dillman, D. (2007). Mail and internet surveys: The tailored design method. New York: Wiley & Sons.
European Statistics Code of Practice—revised edition 2011 http://epp.eurostat.ec.europa.eu/portal/page/portal/product_details/publication?p_product_code=KS-32-11-955
ESS Standard for Quality Report 2009 Edition Eurostat methodologies and working papers.
Eurostat: Directorate B: Quality, Methodology and Information Systems. Editing building block 2011 user’s manual. August 2012.
Eurostat Core variables updated 2011 guidelines. Doc. Eurostat/F/11/DSS/01/2.3EN
Eurostat Tender specification (2011/S 122-201426 and 2011/S 249-404645). http://epp.eurostat.ec.europa.eu/portal/page/portal/calls_for_tenders/archives
Ehling, M. and Körner, T. (eds) (2007). Handbook on data quality assessment methods and Tools. Prepared for Eurostat. http://epp.eurostat.ec.europa.eu/portal/page/portal/quality/documents/HANDBOOK%20ON%20DATA%20QUALITY%20ASSESSMENT%20METHODS%20AND%20TOOLS%20%20I.pdf
Groves, R. (1989). Survey errors and survey costs. New York: Wiley & Sons.
Groves, R. and Peytcheva, E. (2008). The impact of nonresponse rates on nonresponse error: A meta-analysis. Public Opinion Quarterly, 72 (2): 167-189.
Groves, R. and Couper M. (1998). Nonresponse in household interview surveys. New York: Wiley & Sons.
Krosnick, J. A., and Alwin, D. F., (1987). An evaluation of a cognitive theory of response order effects in survey measurement. Public Opinion Quarterly, 51, 201-219.
Meltzer, H. (2008). Report on the development of a survey module on disability and social integration. https://circabc.europa.eu/w/browse/450b8367-d620-4398-844e-07bc5bdf8e6a
Meltzer, H. (2011). Opportunities for statistical indicator development relevant to measuring disability equality in Europe. PowerPoint presentation. ANED Conference, Brussels. http://www.disability-europe.net/content/aned/media/Powerpoint%20Meltzer_Howard_Presentation.pdf
Meltzer, H., Wilmot, A., and Demarest, S. (June 2010) Complementarity of definitions and methods of measuring disability across European surveys. https://circabc.europa.eu/w/browse/268c3432-83ad-4967-9a5e-ea3bb16f5710 (restricted access)
Preferences
47 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
6
Wilmot, A. and Meltzer, H. (September 2010) Report on the development of the European Disability and Social Integration Survey. https://circabc.europa.eu/w/browse/f420b9de-972b-42ca-b929-a585df7c7b43
World Health Organization (WHO). (2001) International classification of functioning, disability and health. http://www.disabilitaincifre.it/documenti/ICF_18.pdf
7 Annexes
48 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
7. ANNEXES Annex A Table A-1: Annex A: Questionnaire Guide
No. Question Description Purpose
Eurostat core social variables
1 Country of Residence Respondent’s country of usual residence (NUTS) Classificatory
2 Region of residence Region where the individual is living (place of usual residence).
Classificatory
3 Degree of urbanisation The type of locality individual is living in, namely whether an urban or a rural area (or a borderline case).
Classificatory
4 Sex Biological sex of person Classificatory
5 Age Age last birthday in completed years Classificatory
6 Legal marital status Legal status of individual in relation to the marriage laws of the country.
Classificatory
7 De facto marital status Marital status of individual in terms of his/her actual living arrangements within the household. Consensual union is defined as the union between non-married partners.
Classificatory
8 Household composition Size and composition of the private household derived from information on relationship between household members and the economic activity status.
Classificatory
9 Education level successfully completed
Highest education level successfully completed. Classificatory
10 Self-declared labour status
Normal or current ‘main’ labour status as perceived by the respondent.
Classificatory
11 Employment status Professional status of employed person (Self-employed/employee)
Classificatory
12 Occupation Occupation in main job Classificatory
13 Economic sector in employment
Economic activity of the local unit where the respondent is employed (incl. Self-employed).
Classificatory
Minimum European Health Module
14 MEHMHS1 General Health Descriptive
15 MEHMHS2 Longstanding illness or health problem Descriptive
16 MEHMHS3 Severity level of longstanding health problem Descriptive
Longstanding health conditions and difficulties with basic activities
17 LHCD Type of health condition or disease Descriptive
18 LDBA Difficulties with basic activities Descriptive
(continued)
Annexes
49 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
7
Table A-1: Annex A: Questionnaire Guide (continued)
No. Question Description Purpose
Mobility
19 MobPrv Barriers to mobility: leaving home Key variable
20 MobPrv1 Lack of special aids or equipment a barrier to mobility
Key variable
21 MobPrv2 Lack of personal help or assistance a barrier to mobility
Key variable
Transport
22 UseVcl Continuous use of household motorised vehicle Filter
23 DrPass Go out in household vehicle as driver or passenger
Filter
24 VclOft Go out in household vehicle less than want to Filter
25 VclPrv Barriers to mobility: household vehicle. Key variable
26 VclPrv1 Lack of special aids or equipment a barrier to mobility
K Key variable
27 VclPrv2 Lack of personal help or assistance a barrier to mobility
Key variable
28 OthTrn Use of other forms of motorised transport Filter
29 OthTOft Use other forms of transport less than want to Filter
30 OthTPrv Barriers to mobility: other forms of transport Key variable
31 OthTPrv1 Lack of special aids or equipment a barrier to mobility
Key variable
32 OthTPrv2 Lack of personal help or assistance a barrier to mobility
Key variable
Accessibility to buildings
33 BldPrv Barriers to accessing buildings Key variable
34 BldPrv1 Lack of special aids or equipment a barrier to accessing buildings
Key variable
35 BldPrv2 Lack of personal help or assistance a barrier to accessing buildings
Key variable
Education and Training
36 Educ Currently studying for a qualification Filter
37 EdPrv Barriers to education and training Key variable
38 EdPrv1 Lack of special aids or equipment a barrier to education and training
Key variable
39 EdPrv2 Lack of personal help or assistance a barrier to education and training
Key variable
(continued)
7 Annexes
50 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
Table A-1: Annex A: Questionnaire Guide (continued)
No. Question Description Purpose
Employment
40 EmpPrv Barriers to employment Key variable
41 EmpPrv1 Lack of special aids or equipment a barrier to employment
Key variable
42 EmpPrv2 Lack of personal help or assistance a barrier to employment
Key variable
Internet use
43 IntUse Uses Internet Filter
44 IntFor Internet purpose Descriptive
45 IntMor Would like to use Internet (more) Filter
46 IntPrv Barriers to using Internet Key variable
47 IntPrv1 Lack of special aids or equipment a barrier to using Internet
Key variable
48 IntPrv2 Lack of personal help or assistance a barrier to using Internet
Key variable
Social contact
49 CloseNum Number of people feel close to Key variable
50 SpkOft Spoke with people feel close to as often as wanted to
Filter
51 SpkPrv Barriers to speaking with people feel close to Key variable
52 SpkPrv1 Lack of special aids or equipment a barrier to speaking with people feel close to
Key variable
53 SpkPrv2 Lack of personal help of assistance a barrier to speaking with people feel close to
Key variable
Leisure pursuits
54 HobPrv Barriers to pursuing hobbies or interests Key variable
55 HobPrv1 Lack of special aids or equipment a barrier to hobbies or interests
Key variable
56 HobPrv2 Lack of personal help of assistance a barrier to hobbies or interests
Key variable
57 EntPrv Barriers to attending cultural events Key variable
58 EntPrv1 Lack of special aids or equipment a barrier to attending cultural events
Key variable
59 EntPrv2 Lack of personal help of assistance a barrier to attending cultural events
Key variable
(continued)
Annexes
51 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
7
Table A-1: Annex A: Questionnaire Guide (continued)
No. Question Description Purpose
Economic life
60 EcPay Ability to pay for the essential things in life Filter
61 EcPrv Barriers to paying for the essential things in life
Key variable
Attitudes and behaviour
62 Discrim Reasons for feeling treated unfairly Key variable
63 DisWho Types of people who treat respondent unfairly Descriptive
Eurostat core social variables (cont.)
64 Country of Citizenship Country of citizenship at time of data collection Classificatory
65 Country of birth Country where respondent was born Classificatory
66 Income Net monthly income of household Classificatory
7 Annexes
52 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
Annex B Table B-1: Comparison of Population Percentages to Weighted and Unweighted Sample Estimates of Demographic Characteristics by Country
Country Characteristics Percent in the
Population
Unweighted Sample
Estimate (%)
Weighted Sample
Estimate (%)
AT Age group
15-19 7 3 7
20-24 7 3 7
25-29 8 3 8
30-34 8 4 8
35-39 8 5 8
40-44 9 8 9
45-49 10 10 10
50-54 9 11 9
55-59 7 10 7
60-64 7 10 7
65-69 6 11 6
70-74 6 11 6
75 years and older 9 10 9
Sex
Male 48 41 48
Female 52 59 52
Degree of urbanization
Densely populated area 30 30 28
Intermediate populated area 29 30 28
Thinly-populated area 41 34 39
Unknown/Missing 6 5
Household composition
1 person 14 24 14
2 persons 24 40 24
3 persons 21 14 21
4 persons 24 14 24
5 persons and more 18 9 18
Highest level of education completed
Primary and lower secondary education (ISCED 1-2)
25 16 25
Upper secondary and post-secondary nontertiary education (ISCED 3-4)
59 62 59
Tertiary education (ISCED 5-6) 16 21 16
(continued)
Annexes
53 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
7
Table B-1: Comparison of Population Percentages to Weighted and Unweighted Sample Estimates of Demographic Characteristics by Country (con’t)
Country Characteristics Percent in the
Population
Unweighted Sample
Estimate (%)
Weighted Sample
Estimate (%)
Federal state
Burgenland 3 3 3
Carinthia 19 19 18
Lower Austria 21 20 19
Upper Austria 7 5 6
Salzburg 14 15 14
Styria 17 17 16
Tyrol 6 5 6
Vorarlberg 9 7 8
Vienna 4 4 4
Missing 7 5
BE Region (EQLS 18+)
Région de Bruxelles-Capitale / Brussels Hoofdstedelijk Gewest
10 14 10
Prov. Antwerpen 16 12 16
Prov. Limburg (BE) 8 12 8
Prov. Oost-Vlaanderen 13 10 13
Prov. Vlaams-Brabant 10 10 10
Prov. West-Vlaanderen 11 9 11
Prov. Brabant Wallon 4 4 4
Prov. Hainaut 12 12 12
Prov. Liège 10 10 10
Prov. Luxembourg (BE) 2 3 2
Prov. Namur 4 4 4
Gender
Male 47 46 47
Female 53 54 53
Age
15-17 4 4 4
18-29 18 19 18
30-39 16 15 16
40-49 18 18 18
50-59 16 16 16
60-69 12 13 12
70+ 15 15 15
Level of Education
ISCED 0-2 34 27 35
(continued)
7 Annexes
54 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
Table B-1: Comparison of Population Percentages to Weighted and Unweighted Sample Estimates of Demographic Characteristics by Country (con’t)
Country Characteristics Percent in the
Population
Unweighted Sample
Estimate (%)
Weighted Sample
Estimate (%)
ISCED 3-4 36 35 36
ISCED 5-6 30 38 30
BG Age
15-24 14 10 14
25-59 57 53 57
60-74 20 26 20
75+ 9 12 9
Sex
Male 48 48 48
Female 52 52 52
Region of residence
North-West 12 12 12
North-Central 12 11 11
North-East 13 13 13
South-East 14 15 15
South-West 29 28 29
South-Central 20 21 20
Degree of urbanisation
Densely-populated area 43 43 44
Intermediate populated area 23 24 23
Thinly-populated area 34 33 33
Household type
One-person households n/a 22 21
Lone parent with child(ren) aged less than 25
n/a 3 4
Couple without child(ren) aged less than 25
n/a 27 24
Couple with child(ren) aged less than 25
n/a 22 25
Couple or lone parent with child(ren) aged less than 25 and
n/a 8 9
Other type of household n/a 18 18 Highest level of education
completed
No formal education or below ISCED 1
2 1 1
Primary education 5 4 4
Lower secondary education 24 20 20
Upper secondary education 47 54 55
Tertiary education 21 20 20
(continued)
Annexes
55 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
7
Table B-1: Comparison of Population Percentages to Weighted and Unweighted Sample Estimates of Demographic Characteristics by Country (con’t)
Country Characteristics Percent in the
Population
Unweighted Sample
Estimate (%)
Weighted Sample
Estimate (%)
CY Age
15-24 18 24 30
25-34 25 18 15
35-44 21 16 14
45-54 20 19 20
55-64 16 23 20 Sex
Male 47 41 43
Female 53 59 57
Degree of urbanisation
Densely-populated area 55 53 51
Intermediate urbanised area 19 23 23
Thinly-populated area 26 24 26
Household Composition
Single adult without children 8 15 6
More than one persons 92 85 94
Education level (15-74)
ED0-2 30 33 31
ED3_4 37 39 41
ED5_6 33 29 27
CZ Sex, GP15+
Male 49 46 49
Female 51 54 51
Age Categories, GP15+
15-24 14 12 15
25-34 17 16 14
35-44 18 19 20
45-54 15 14 18
55-64 17 16 15
65+ 19 23 19
Region, GP15+
CZ01 12 12 12
CZ02 12 12 12
CZ03 12 11 12
CZ04 11 11 11
CZ05 14 14 14
CZ06 16 16 16
CZ07 12 12 12
(continued)
7 Annexes
56 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
Table B-1: Comparison of Population Percentages to Weighted and Unweighted Sample Estimates of Demographic Characteristics by Country (con’t)
Country Characteristics Percent in the
Population
Unweighted Sample
Estimate (%)
Weighted Sample
Estimate (%)
CZ08 12 12 12
Degree of urbanization, GP15+ (preliminary census data)
Densely-populated area 31 29 29
Intermediate populated area 32 34 34
Thinly-populated area 37 37 37
Nr of persons in private households, GP total
1 person 14 23 13
More than 1 person 86 77 87 Education, GP15+
Low 54 46 54
Medium 32 40 32
High 15 14 15
EE Age
15-24 15 13 15
25-39 26 26 26
40-59 32 32 32
60-74 18 16 18
75 and older 10 10 10
Gender
Female 55 57 55 Male 45 44 45
Region NUTS 1 = Estonia, 100% Degree of urbanization
Densely populated area 42 44 42
Intermediate populated area 17 17 17
Thinly-populated area 41 39 41
Household composition
One-person household 36 21 25
Multi-person household 64 79 75 Highest level of educationa
completed
ISEC 1—primary 18 3 2
ISEC 2—lower secondary 10 10
ISEC 3—upper secondary 51 32 34
ISEC 4—post secondary 25 25
ISEC 5—tertiary, first stage 31 30 29
ISEC 6—tertiary, second stage 1 1
(continued)
Annexes
57 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
7
Table B-1: Comparison of Population Percentages to Weighted and Unweighted Sample Estimates of Demographic Characteristics by Country (con’t)
Country Characteristics Percent in the
Population
Unweighted Sample
Estimate (%)
Weighted Sample
Estimate (%)
FR Sex
Male 48 43 48
Female 52 57 52
Age
15-24 15 11 15
25-34 16 14 14
35-44 18 17 18
45-54 17 16 18
55-64 15 19 16
65+ 20 23 20
Region
FR1 19 15 18
FR2 17 17 17
FR3 6 6 6
FR4 9 9 9
FR5 14 15 13
FR6 11 12 11
FR7 12 13 12
FR8 13 11 13
Degree of Urbanisation
Densely-populated area 46 43 45
Intermediate populated area 21 21 21
Thinly-populated area 33 35 33
Number of persons in the household
Only 1 person in the household 18 28 18
More than one person 82 72 82
Education
ISCED 0-2 37 26 37
ISCED 3-4 40 41 40
ISCED 5-6 23 32 23
Refusal/Don’t Know 0.6 0.6
DE Age
15-19 6 5 6
20-24 7 5 7
25-29 7 6 7
30-34 7 6 7
35-39 7 7 7
(continued)
7 Annexes
58 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
Table B-1: Comparison of Population Percentages to Weighted and Unweighted Sample Estimates of Demographic Characteristics by Country (con’t)
Country Characteristics Percent in the
Population
Unweighted Sample
Estimate (%)
Weighted Sample
Estimate (%) 40-44 9 9 9
45-49 10 11 10
50-54 9 11 9
55-59 8 10 8
60-64 7 9 7
65-69 6 8 6
70-74 7 7 7
75 years and older 11 8 11
Sex
Male 49 45 49
Female 51 55 51
Federal state
Schleswig-Holstein (Nuts-1: DEF)
4 3 3
Hamburg (Nuts-1: DE6) 2 2 2
Niedersachsen (Nuts-1: DE9) 10 9 9
Bremen (Nuts-1: DE5) 1 1 1
Nordrhein-Westfalen (Nuts-1: DEA)
22 18 21
Hessen (Nuts-1: DE7) 7 7 7
Rheinland-Pfalz (Nuts-1: DEB) 5 4 5
Baden-Württemberg (Nuts-1: DE1)
13 11 13
Bayern (Nuts-1: DE2) 15 16 15
Saarland (Nuts-1: DEC) 1 1 1
Berlin (Nuts-1: DE3) 4 5 4
Brandenburg (Nuts-1: DE4) 3 4 3
Federal state (continued)
Mecklenburg-Vorpommern (Nuts-1: DE8)
2 3 2
Sachsen (Nuts-1: DED) 5 6 5
Sachsen-Anhalt (Nuts-1: DEE) 3 3 3
Thüringen (Nuts-1: DEG) 3 3 3
unknown/Missing 3 4
Degree of urbanization
Densely populated area 35 37 34
Intermediate populated area 42 39 40
Thinly-populated area 23 20 22
Unknown/Missing 4 5
HH composition
1 person 22 26 22
(continued)
Annexes
59 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
7
Table B-1: Comparison of Population Percentages to Weighted and Unweighted Sample Estimates of Demographic Characteristics by Country (con’t)
Country Characteristics Percent in the
Population
Unweighted Sample
Estimate (%)
Weighted Sample
Estimate (%)
2 persons 38 37 38
3 persons 18 17 18
4 persons 16 14 16
5 persons and more 7 6 7
Highest level of education completed
Hauptschulabschluss, POS 8 44 20 44
Realschulabschluss, POS 10 22 35 22
Fachhochschulreife, Hochschulreife
27 44 27
kein Schulabschluss, noch in Ausbildung
8 2 8
EL Age
15-24 15 18 18
25-34 21 19 19
35-44 24 24 23
45-54 22 22 22
55-64 19 18 18
Sex
Male 49 42 43
Female 51 58 57
Degree of Urbanization
Densely-populated area 42 42 41
Intermediate urbanised area 28 26 26
Thinly-populated area 30 32 33
Household Composition
Single adult without children 13 20 9
More than one person 87 79 91
Education Level
ED0-2 41 36 42
ED3-4 38 49 44
ED5-6 21 16 14
Region
1 32 32 32
2 22 22 22
3 36 35 36
4 10 10 10
HU Age
15-24 14 16 14
25-34 17 17 17
(continued)
7 Annexes
60 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
Table B-1: Comparison of Population Percentages to Weighted and Unweighted Sample Estimates of Demographic Characteristics by Country (con’t)
Country Characteristics Percent in the
Population
Unweighted Sample
Estimate (%)
Weighted Sample
Estimate (%) 35-44 17 18 17
45-54 15 15 15
55-64 16 15 16
65-74 11 11 11
75+ 9 8 9
Sex
Male 47 47 47
Female 53 53 53
NUTS1
HU1 30 28 30
HU2 30 30 30
HU3 39 43 39
Size of household
1 11 15 11
2 28 28 28
3 23 23 23
4-x 39 34 39
IS Region (Eurostat 15+) Höfudborgarsvædi 64 65 64
Landsbyggd 36 35 36
Gender
Male 49 49 49
Female 51 51 51
Age
15-17 6 6 6
18-29 22 22 22
30-39 18 17 18
40-49 17 17 17
50-59 16 16 16
60-69 11 11 11
70+ 11 11 11
Level of Education
ISCED 0-2 37 29 37
ISCED 3-4 35 37 35
ISCED 5-6 27 34 27
(continued)
Annexes
61 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
7
Table B-1: Comparison of Population Percentages to Weighted and Unweighted Sample Estimates of Demographic Characteristics by Country (con’t)
Country Characteristics Percent in the
Population
Unweighted Sample
Estimate (%)
Weighted Sample
Estimate (%)
IT Age Categories, GP15+
15-24 12 11 12
25-34 14 14 13
35-44 18 19 19
45-54 17 15 17 55-64 15 13 14
65+ 24 28 24
Sex, GP15+
Male 48 45 48
Female 52 55 52
Region, GP15+
ITC1 Piemonte 7 7 7
ITC2 Valle d’Aosta 0 0 0
ITC3 Liguria 3 3 3
ITC4 Lombardia 16 17 16
ITF1 Abruzzo 2 2 2 Region, GP15+ (continued)
ITF2 Molise 1 1 1
ITF3 Campania 9 10 10
ITF4 Puglia 7 7 7
ITF5 Basilicata 1 1 1
ITF6 Calabria 3 3 3
ITG1 Sicilia 8 8 8
ITG2 Sardegna 3 3 3
ITH1 Provincia Autonoma di Bolzano
1 1 1
ITH2 Provincia Autonoma di Trento
1 1 1
ITH3 Veneto 8 8 8
ITH4 Friuli-Venezia Giulia 2 2 2
ITH5 Emilia-Romagna 7 7 7 ITI1 Toscana 6 6 6
ITI2 Umbria 2 1 2
ITI3 Marche 3 3 3
ITI4 Lazio 9 9 9
Degree of urbanization
Densely-populated area 33 34 34
Intermediate populated area 42 43 43
Thinly-populated area 25 23 23
(continued)
7 Annexes
62 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
Table B-1: Comparison of Population Percentages to Weighted and Unweighted Sample Estimates of Demographic Characteristics by Country (con’t)
Country Characteristics Percent in the
Population
Unweighted Sample
Estimate (%)
Weighted Sample
Estimate (%)
Nr of persons by size of household
1 person 10 28 22
More than 1 person 90 72 78
Education, GP15+
Low 53 51 53
Middle 35 38 35
High 12 11 12
LV Age
15-19 6 13 6
20-24 8 26 8
25-29 9 4 9
30-34 8 4 8
35-39 8 5 8
40-44 8 6 8
45-49 8 7 8
50-54 9 9 9
55-59 8 9 7
60-64 7 7 7
65-69 6 2 6
70-74 6 3 6
75+ 10 5 10
Sex
Male 45 44 44
Female 55 56 56
Region (NUTS 1 level)
Latvia 100 100 100
Degree of urbanization
DEGURBA 1 n/a 36 42
DEGURBA 2 n/a 24 20
DEGURBA 3 n/a 40 38
Household composition
1 person 18 14 15
More than 1 person 83 86 85
Highest level of education completed
Low 23 14 22
Medium 53 54 55
High 23 32 23
(continued)
Annexes
63 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
7
Table B-1: Comparison of Population Percentages to Weighted and Unweighted Sample Estimates of Demographic Characteristics by Country (con’t)
Country Characteristics Percent in the
Population
Unweighted Sample
Estimate (%)
Weighted Sample
Estimate (%)
LT Age
15-24 21 19 21
25-34 19 20 19
35-44 20 19 20
45-54 23 27 27 55-64 18 15 13
Sex
Male 45 45 48
Female 55 55 52
Degree of urbanization
Densely-populated area 43 42 41
Intermediate urbanized area 10 11 10
Thinly-populated area 48 47 49
Household Composition
Single adult without children 18 20 10
More than one persons 82 80 90 Level of Education
ED0-2 18 16 20
ED3_4 55 58 56
ED5_6 27 25 24
LU Age
15-19 7 5 7
20-24 7 3 7
25-29 9 5 9
30-34 9 7 9
35-39 9 9 9
40-44 10 12 10
45-49 10 11 10 50-54 9 11 9
55-59 7 10 7
60-64 6 8 6
65-69 6 7 6
70-74 4 5 4
75 and older 6 6 6
Sex
Male 50 43 50
Female 50 57 50
(continued)
7 Annexes
64 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
Table B-1: Comparison of Population Percentages to Weighted and Unweighted Sample Estimates of Demographic Characteristics by Country (con’t)
Country Characteristics Percent in the
Population
Unweighted Sample
Estimate (%)
Weighted Sample
Estimate (%)
Degree of urbanization
Densely populated area 18 15 17
Intermediate populated area 37 31 34
Thinly-populated area 45 46 42
Unknown/Missing 8 8
Household composition
1 Person 12 24 12
2 Persons 23 28 23
3 Persons 20 18 20
4 Persons 26 19 26
5 Persons and more 19 11 19
Highest level of education completed
Primary and lower secondary education (ISCED 1-2)
30 24 30
Upper secondary and postsecondary nontertiary education (ISCED 3-4)
41 41 40
Tertiary education (ISCED 5-6) 29 35 29
Unknown/Missing 1 1
MT Age
15-19 7 7 7
20-29 17 16 17
30-39 17 14 17
40-49 15 15 15
50-59 17 17 17
60-69 16 19 16
70+ 12 13 12
Sex
Male 50 49 50
Female 50 51 50
Degree of urbanization
2 100 100 100
3 0 0 0
Type of household
1 7 8 7
2 3 3 3
3 18 20 18
4 38 36 38
5 2 2 2
6 31 31 31
-1 0 0 0
(continued)
Annexes
65 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
7
Table B-1: Comparison of Population Percentages to Weighted and Unweighted Sample Estimates of Demographic Characteristics by Country (con’t)
Country Characteristics Percent in the
Population
Unweighted Sample
Estimate (%)
Weighted Sample
Estimate (%)
Education
0 2 3 2
1 20 22 20
2 36 36 36
3 19 18 19
4 9 9 9
5 11 11 11
6 1 1 1
-1 0 0 0
NL Region (EQLS 18+)
Groningen 3 4 3
Friesland (NL) 4 3 4
Drenthe 5 3 5
Overijssel 7 6 7
Gelderland 2 2 2
Flevoland 12 13 12
Utrecht 7 8 7
Noord-Holland 11 12 11
Zuid-Holland 14 15 14
Zeeland 2 2 2
Noord-Brabant 15 14 15
Limburg (NL) 7 7 7
A’dam, R’dam en Den Haag 11 12 11
Gender
Male 49 47 49
Female 51 53 51
Age
15-17 4 2 4
18-29 18 16 18
30-39 16 17 16
40-49 19 20 19
50-59 17 18 17
60-69 14 14 14
70+ 12 13 13
Level of Education
ISCED 0-2 33 26 33
ISCED 3-4 39 29 39
ISCEd 5-6 28 45 28
(continued)
7 Annexes
66 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
Table B-1: Comparison of Population Percentages to Weighted and Unweighted Sample Estimates of Demographic Characteristics by Country (con’t)
Country Characteristics Percent in the
Population
Unweighted Sample
Estimate (%)
Weighted Sample
Estimate (%)
Household Size
1 person 36 28 34
2 people 33 33 33
3 people 12 13 13
4 or more people 19 25 20
NO Sex
Male 50 50 50
Female 50 51 50
Age
15-24 16 13 16
25-34 16 9 16
35-44 18 16 18
45-54 17 22 17
55-64 15 20 15
65+ 19 20 19
Household Size
1 person 18 21 19
2 or more 82 79 81
Education
Lower 28 14 17
Medium 41 33 45
Higher 31 53 38
Urbanization
Densely 27 27 27
Intermediate 34 34 33
Thinly populated 39 39 39
PT Age
15-24 13 14 18
25-34 16 11 16
35-44 18 14 16
45-54 17 14 19
55-64 15 17 14
65+ 23 30 18
Sex
Male 47 43 49
Female 53 57 51
Region of residence
Norte 37 33 31
(continued)
Annexes
67 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
7
Table B-1: Comparison of Population Percentages to Weighted and Unweighted Sample Estimates of Demographic Characteristics by Country (con’t)
Country Characteristics Percent in the
Population
Unweighted Sample
Estimate (%)
Weighted Sample
Estimate (%) Centro 23 27 19
Lisboa 28 25 36
Alentejo 7 10 8
Algarve 4 5 5
Degree of urbanization
Densely-populated area 43 39 51
Intermediate populated area 29 34 25
Thinly-populated area 28 28 24
Number of person(s) in the household
1 person 8 19 3
More than 1 person 92 81 97
Highest level of education completed
Low 69 81 68
Medium 17 11 19
High 14 8 14
RO Age 15-24 15 7 11
25-34 19 9 12
35-44 20 14 17
45-54 14 18 24
55-64 15 20 18
Sex
Female 52 52 49
Male 48 48 51
Region (NUTS 1 level)
Macroregion RO1 24 25 24
Macroregion RO2 30 29 30
Macroregion RO3 26 25 26
Macroregion RO4 20 21 20
Degree of urbanization
Density populated area No Data 34 35
Intermediate area No Data 21 23
Thinly populated area No Data 45 42
Household composition
1 person No Data 29 9
More than 1 person No Data 71 91
(continued)
7 Annexes
68 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
Table B-1: Comparison of Population Percentages to Weighted and Unweighted Sample Estimates of Demographic Characteristics by Country (con’t)
Country Characteristics Percent in the
Population
Unweighted Sample
Estimate (%)
Weighted Sample
Estimate (%)
Highest level of education completed
Low No Data 36 30
Medium No Data 53 58
High No Data 11 12
SI Age
15-24 No data 13 13
25-34 No data 15 17
35-44 No data 16 17
45-54 No data 17 18
55-64 No data 17 16
65-74 No data 13 11
75-84 No data 8 7
85+ No data 2 2
Sex
Male No data 46 49
Female No data 54 51
Statistical Regions (NUTS-3 Level)
Gorenjska No data 11 10
Goriska No data 7 6
Jugovzhodna Slovenija No data 6 7
Koroska No data 6 4
Notranjsko-kraska No data 2 3
Obalno-kraska No data 5 6
Osrednjeslovenska No data 20 26
Podravska No data 16 16
Pomurska No data 7 6
Savinjska No data 14 13
Spodnjeposavska No data 4 3
Zasavska No data 2 2
Degree of urbanization
Densely populated area No data 12 19
Intermediate density area No data 33 31
Thinly-populated area No data 56 50
Highest level of education completed
Below upper secondary education
No data 23 29
Upper secondary education No data 56 53
Tertiary education No data 21 18
(continued)
Annexes
69 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
7
Table B-1: Comparison of Population Percentages to Weighted and Unweighted Sample Estimates of Demographic Characteristics by Country (con’t)
Country Characteristics Percent in the
Population
Unweighted Sample
Estimate (%)
Weighted Sample
Estimate (%)
ES Age
15-24 No Data 8 12
25-34 No Data 12 17
35-44 No Data 21 20
45-54 No Data 19 18
55-64 No Data 15 14
65+ No Data 25 20
Sex
Female No Data 54 51
Male No Data 46 49
Region
1 No Data 14 10
2 No Data 17 10
3 No Data 10 14
4 No Data 16 12
5 No Data 22 29
6 No Data 18 21
7 No Data 4 5
Degree of urbanization
Densely No Data 48 49
Intermediate No Data 24 25
Thinly No Data 29 26
Household composition
One person No Data 21 9
Lone parent with ch. No Data 5 5
Couple without ch. No Data 22 19
Couple with child No Data 31 37
Lone/coup + other No Data 5 9
Other No Data 17 21
Education
Low No Data 55 54
Medium No Data 19 20
High No Data 26 26
SE Region (EQLS 18+)
Stockholm 22 21 22
Ostra Mellansverige 17 17 17
Småland med öarna 9 9 9
Sydsverige 15 15 15
(continued)
7 Annexes
70 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
Table B-1: Comparison of Population Percentages to Weighted and Unweighted Sample Estimates of Demographic Characteristics by Country (con’t)
Country Characteristics Percent in the
Population
Unweighted Sample
Estimate (%)
Weighted Sample
Estimate (%)
Västsverige 20 20 20
Norra Mellansverige 9 9 9
Mellersta Norrland 4 4 4
Övre Norrland 5 5 5
Gender
Male 48 48 48
Female 52 52 52
Age 15-17 5 5 5
18-29 19 19 19
30-39 16 16 16
40-49 17 17 17
50-59 15 15 15
60-69 15 15 15
70+ 15 15 15
Level of Education
ISCED 0-2 25 18 24
ISCED 3-4 50 41 50
ISCED 5-6 26 41 26
UK Age
15-24 16 6 16
25-34 16 10 16
34-44 17 15 17
45-54 17 19 17
55-64 15 20 14
Sex
Female 51 57 51
Male 49 43 49
Region (NUTS1 level) UKC 4 4 3
UKD 11 9 9
UKE 8 7 7
UKF 7 5 6
UKG 9 7 7
UKH 9 7 7
UKI 13 8 13
UKJ 14 11 10
(continued)
Annexes
71 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
7
Table B-1: Comparison of Population Percentages to Weighted and Unweighted Sample Estimates of Demographic Characteristics by Country (con’t)
Country Characteristics Percent in the
Population
Unweighted Sample
Estimate (%)
Weighted Sample
Estimate (%)
UKK 9 7 6
Region (NUTS1 level) (continued)
UKL 5 4 4
UKM 9 10 8
UKN 3 3 3
Degree of urbanisation
Densely populated area 58 42 48
Intermediate area 29 24 22
Thinly populated area 13 13 10
Household Composition
One-person household No Data 35 24
Lone parent with child(ren) aged less than 25
No Data 4 4
Couple without child(ren) aged less than 25
No Data 27 23
Couple with child(ren) aged less than 25
No Data 14 18
Couple or lone parent with child(ren) aged less than 25 and other persons living in household
No Data 2 3
Other type of household No Data 17 25
Refusal or Don’t Know No Data 2 3
Highest level of education completed
No formal education or below ISCED 1
No Data 2 1
ISCED 1—Primary education No Data 1 1
ISCED 2—Lower secondary education
No Data 13 11
ISCED 3—Upper secondary education
No Data 33 36
ISCED 4—Postsecondary education but not in tertiary
No Data 2 2
ISCED 5—Tertiary education, first stage
No Data 43 43
ISCED 6—Tertiary education, second stage
No Data 2 2
Refusal or Don’t Know No Data 4 4 a Population values based on 15-74 year olds
7 Annexes
72
Annex C Table C-1: Question Modifications and Problems by Country
Name of Country Sequence Followed Yes/No
Questions Modified Yes/No
Modifications Made Problematic Questions
Austria Y N N/A None reported
Belgium Y N ▪ When the selected respondent was 15 to 17 years old, an additional question was included, asking for their parent’s permission for their participation in the research.
▪ None—there were no comprehension problems with the questionnaire.
Bulgaria Y Y ▪ Household type (sector Household composition) was modified in the Bulgarian version as in the predominant part of homes (houses or flats/apartments) live two or three generations together and the given in the questionnaire break down/split covers only part of family members combinations. At about 18% of all respondents are in “Other” answer.
▪ Classificatory variables could cause a problem with the quality of the received data as this part of the questionnaire was provided only as indicators. Thanks to the coordinator for Bulgaria, we learned for availability of the Core variables manual. Instructions in this manual helped us to prepare the proper questions and instructions for them.
▪ In some towns people were not inclined to give information about their date of birth. To identify the respondent, the household members were asked to tell at least the month of birth and the age in completed years.
Cyprus Y N N/A None reported
Czech Republic Y Y ▪ Age of children of less than one year was recorded as 1 (household roster)
▪ LHCD: “NONE” was added as a spontaneous response category (i.e., not read out by interviewers)
▪ Most respondents had issues understanding the concept of “basic activities”; translated into Czech, this concept does not realty make sense. It would have been better to read out the question without phrasing “basic activities,” and so just read out “for example, seeing, hearing, concentration and moving around.”
(continued)
7 Annexes
73EU Comparative Quality and Technical Report: European Health and Social Integration Survey
Table C-1: Question Modifications and Problems by Country (continued)
Name of Country Sequence
Followed Yes/No
Questions Modified Yes/No
Modifications Made Problematic Questions
Denmark Y Y ▪ Some questions were slightly modified due to the method used in collection of data (telephone and web instead of face-to-face interviews). We skipped the last sentence from the question LHCD (skipped=bold): May I just check, do you have any of the following longstanding health conditions, illnesses or diseases? Please use this card as a guide and indicate all that apply. As no interviews were conducted face-to-face, it made no sense to the respondent to read or hear this last sentence.
▪ The following questions from the original questionnaire were not suited for telephone interviews: EdPrv12, EmpPrv12, HobPrv11, EntPrv11. E.g., EmpPrv12: Don’t want paid work. This question does not work well in a telephone interview and therefore it was modified to a question: Do you want a paid job? The question was asked in the same order as in the original questionnaire. It should be noticed that the original filter was kept and both employed and unemployed were asked the question.
▪ We added an extra text before question LHCD: MEHMHS2: Do you have any longstanding illness or longstanding health problem? By longstanding I mean illnesses or health problems which have lasted, or are expected to last, for 6 months or more. Followed by: MEHMHS3: For at least the last 6 months, to what extent have you been limited because of a health problem, in activities people usually do? Would you say you have been. In the original questionnaire LHCD we now asked: May I just check, do you have any of the following longstanding health conditions, illnesses or diseases? With 18 areas for the respondent to be asked regardless that some respondents already had answered ‘no’ to any longstanding illness or health problems in question MEHMHS2. The text we added came before LHCD and goes: Even though you have answered that you do not have any longstanding illness or health problems we would like to ask you some thoroughly questions as some illnesses or health problems are often neglected.
See previous column
(continued)
7 Annexes
74 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
Table C-1: Question Modifications and Problems by Country (continued)
Name of Country Sequence Followed Yes/No
Questions Modified Yes/No
Modifications Made Problematic Questions
Estonia Y N N/A ▪ Some respondents found the question LHCD (asking about having different longstanding health conditions or diseases) to be too personal and irritating. Some resistance was also met in relation to the questions on household composition.
▪ Respondents found the questions on issues that they did not have problems with not concerning them; for example some respondents found the questions on mobility to be humorous in relation to their lives. In those cases it was possible to explain why there is a need to ask different kinds of questions from all respondents.
Finland Y N N/A ▪ EdPrv, option 7, “Difficulties in getting to learning facility.” Some have wondered if this meant physical accessing to a learning facility, or difficulties being admitted to a school. This was somewhat ambiguous in Finnish translation. The former was the correct, this was explained in the interviewer instructions, thus no corrections to the questionnaire were needed.
France Y N N/A ▪ To ensure concept understood correctly. Issues with IntoEmp being difficult to translate into French.
Germany Y N N/A None
(continued)
7 Annexes
75EU Comparative Quality and Technical Report: European Health and Social Integration Survey
Table C-1: Question Modifications and Problems by Country (continued)
Name of Country Sequence Followed
Yes/No
Questions Modified Yes/No
Modifications Made Problematic Questions
Greece Y Y ▪ In HATLEVEL we have used code 0 for those who have not completed a level of education.
▪ In JOBISCO we have insert the categories that we have found at internet for Greece(ISCO 08).
▪ We have improved the translation for EHISHA1 code4,6;
▪ EmpPrv; IntroSoc and CloseNum; IntroEc and EcPay; IntroDis.
▪ In HHINCOME the addition of a scale was deemed necessary and has been added.
▪ Overall there weren’t major problems with the questionnaire, since the usage of CAPI enabled us to avoid routing and other errors.
▪ Nevertheless one thing that attracted our attention was the percentage of “other” option in some questions (e.g., intfor,ecprv). Maybe in a future wave the “other” should be open-ended (better checking, greater knowledge).
Hungary Y N N/A None reported
Iceland Y N N/A None reported
Italy Y Y ▪ Age of children of less than 1 year old was recorded as 1 (household roster)
▪ LHCD: “NONE” was added as a spontaneous response category (i.e., not read out by interviewers).
▪ Household roster: marital status for all household members aged 15 and above; solution for future rounds: use a simplified household roster.
▪ Level of education: older respondents had difficulties “translating” their level of education into the pre-coded list of educational categories
▪ ISCO-code (profession) and NACE-code (sector of activity): respondents found it difficult to provide enough detailed information that would allow to select the correct classification code.
Latvia Y N N/A ▪ Mainly translation issues.
(continued)
7 Annexes
76 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
Table C-1: Question Modifications and Problems by Country (continued)
Name of Country Sequence Followed Yes/No
Questions Modified Yes/No
Modifications Made Problematic Questions
Lithuania Y N N/A ▪ EHISPC2 EHISHA3—Wider explanation of “lack of special aids or equipment” (in Lithuanian—specialių priemonių arba įrangos trūkumas) was needed.
▪ MobPrv—Question on “leaving your home.” Is it leaving your home to go to work or on vacation? Solution to the problem—Explanation of “Leaving” meaning
▪ EcPrv:The ambiguity of the word “essential things.” Wider explanation was needed.Solution to the problem—Explanation of “Essential things” meaning (in Lithuanian—”pagrindinės reikmės”).
Luxembourg Y N N/A None reported
Malta Y N N/A None reported
Netherlands Y N N/A None—there were no comprehension problems with the questionnaire.
Norway Y Y ▪ De facto marital status: In an earlier version of this question, we included an answer code saying ‘SPONTANEOUS ONLY Same sex couple (but not in a formal registered Civil Partnership).’ We have removed this though as we do not need it in order to provide you with the answer codes you have asked for in the transmission guidelines.
▪ HHLABOUR_EMP and HHLABOUR_NEMP are set as one question in our questionnaire (HHLABOUR_EMP)—this will be redefined to original variables in the data file.
▪ HATLEVEL is changed so that it is suitable for the Norwegian standards.
▪ HHINCOME is changed so that it is suitable for Norwegians from Net.
▪ Monthly Income to Gross Annual Income. The Gross Annual Income is included in our datafiles.
▪ All the xx_PRV questions were causing fatigue both for the respondent and the interviewer. It was quite a difficult task to get the respondents to not just answer what they remembered/ assumed, but get the respondent to listen to all the statements per PRV. The questions were not regarded as “sensitive,” so it was the magnitude of statements, not the content that was perceived as challenging.
(continued)
7 Annexes
77EU Comparative Quality and Technical Report: European Health and Social Integration Survey
Table C-1: Question Modifications and Problems by Country (continued)
Name of Country Sequence Followed
Yes/No
Questions Modified Yes/No
Modifications Made Problematic Questions
Poland Y N N/A EmpPrv, when asked retired or pensioners. There should be answer option “retired / pensioned” because the choice “don’t want paid job” although formally accurate can be misleading or perceived as insulting by that persons.
Portugal Y N N/A Income and Household composition caused some problems with respondents.
Romania Y Y ▪ The Question coded MAINSTAT (self-declared labor status) was modified: we removed the answer in compulsory military service as it no longer applies to Romania.
▪ The answers to the ISCO (Occupation in employment) and NACE (Economic sector in employment) questions were required in 4 digits codes and we later recoded them.
▪ Questions 43 and 44 (variable EmpPrv) from the adult questionnaire should have had separate responses and show cards too, and not a shared one (even though the response categories were identical).
▪ Some persons were apprehensive about answering the income section (especially people with higher incomes).
▪ Some questions in the sections referring to Education and Training and Employment were inadequate for older persons (retired persons) and should have not applied to them.
Slovak Republic Y N N/A ▪ There were small problems with wording in some questions but these problems were solved after pilot interviews were done: for example in the household Composition‟ part, the respondents had the tendency to quote the number of household members while excluding themselves. Therefore the question wording also included the phrase ”..members including yourself...‟. Slovak words: “vrátane vás” (which means “including you”).
(continued)
7 Annexes
78 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
Table C-1: Question Modifications and Problems by Country (continued)
Name of Country
Sequence Followed Yes/No
Questions Modified Yes/No
Modifications Made Problematic Questions
Slovenia Y Y ▪ One statement was added in the VCLPRV question: statement nr. 7 (in questionnaire in national language) “I do not have anyone to take me.”
▪ In several questions respondents had the possibility to report that “Other reasons” prevent them from different activities. In the Slovene version of questionnaire we asked respondents to specify the reasons (open-ended question) if they answered “Yes” at statements “Other reasons.”
Spain Y N N/A ▪ The elderly have trouble the following sections: employment; education. Instructions and materials were given to the interviewers during the training in order to solve this kind these of problems during data collection. They were also given instructions to encourage the interviewed to answer in case of refusal.
▪ It is not clear for some respondents the meaning of ‘special aids’
▪ The respondents reluctantly answer the following topics: Household income; Occupation in their job; The discrimination section. One idea to get information about sources of income and total amount of income for the next round is to use administrative sources. During the preparation of the questionnaire we modified the question household composition to simplify the data collection of members of the household.
Sweden Y N N/A ▪ None—there were no comprehension problems with the questionnaire.
(continued)
Table C-1: Question Modifications and Problems by Country (continued)
Name of Country Sequence Followed Yes/No
Questions Modified Yes/No
Modifications Made Problematic Questions
United Kingdom N Y ▪ Following the cognitive testing of the questionnaire, the following changes were made to the wording of questions in the main body of the questionnaire, as detailed in the Cognitive Interview Report and agreed with the European Health and Social Integration Survey (EHSIS) coordinator.
▪ The wording, sentence structure and grammar of some of the answer codes was tweaked so that they flowed more easily when being read out to respondents. This was necessary because, as the survey was conducted over the telephone rather than face-to-face, the answer codes were read to respondents by interviewers rather than presented on a show card for respondents to read themselves. It was ensured that the meaning of the questions was not changed. Some specific examples are as follows:
▪ LHCD
– ‘Problems with arms or hands (include arthritis or rheumatism)’ was changed to ‘Problems with your arms or hands (including arthritis or rheumatism)’.
▪ LDBA
– ‘Using hands and fingers, such as picking up small objects, for example, a button or pencil, or opening or closing containers or bottles’ was changed to ‘Using hands or fingers, such as picking up small objects like a button or a pencil, or opening or closing containers or bottles’.
▪ LDBA, EHISPC1, EHISHA1
– The words ‘Unable to do’ were removed from ‘Cannot do at all / Unable to do’.
▪ EHISPC1
– ‘Getting in and out of bed or chair’ was changed to ‘Getting in and out of your bed or a chair’.
▪ EmpPrv
– ‘Affects receipt of benefits or services’ was changed to ‘It would affect the benefits or services you receive’.
▪ Discrim
– Interviewers were asked not to read out the answer code ‘None of these’. This is because, if a respondent did not answer yes to any of the other answer codes at this question, the interviewer would know to code the respondent as ‘None of these’ without reading it out.
▪ As described above, some respondents found the questionnaire long and repetitive, which irritated them at times. Examples of questions that respondents found particularly repetitive are provided below:
– HobPrv, HobEquip and HobPhelp were perceived as very similar to EntPrv, EntEquip and EntPhelp. In a future survey, it might be worth including some sentences before EntPrv for interviewers to read out which explain what the next three questions are about and how they differ from the previous three questions.
– At EdPrv, EmpPrv, SpkPrv, HobPrv and EntPrv respondents are asked about difficulties accessing or using buildings, when this issue has already been covered in detail at BldPrv, BldEquip and BldPhelp.
– If respondents said ‘No’ at MEHMHS2 and ‘Not limited at all’ at MEHMHS3, they were sometimes irritated to be asked LHCD. A similar point was raised in our Cognitive Interview Report, following which it was understood that these questions had to be asked. However, in any future surveys, it might be worth thinking about different ways to ask these questions.
– The xxxEquip and xxxPhelp were seen as particularly repetitive.
– The long lists of answer codes at the xxPrv questions allowed respondents the opportunity to lose concentration and stop listening.
7 Annexes
80 EU Comparative Quality and Technical Report: European Health and Social Integration Survey
top related