Reaching the hard-to-reach

The cross-national sampling of migrants through Facebook advertisements

InGRID-2 Expert Workshop “Methods and data infrastructure to measure the quality of life of various vulnerable groups”,

Budapest (April 26, 2018).

Steffen Pötzschke

Structure

1. Sampling migrants (in cross-national research)

2. Study design

3. Results

4. Conclusions

2

1. Sampling migrants (in cross-national research)

2. Study design

3. Results

4. Conclusions

3

Structure

There are several established techniques to sample migrants; however, they all suffer from shortcomings. For example:

Use of population registers: Limited to ‘legal’ residents;

Researchers are not always granted (unrestricted) access and registers might not contain all needed variables (Groenewold & Lessard-Phillips, 2012);

Do not exist in all countries.

Random-route sampling: Restriction of the sampling to specific areas;

Cost-intensive (due to the interviewers needed ‘on the ground’ in each location);

Becomes increasingly ineffective the smaller a target population is compared to the overall population (Salentin, 2014).

4

1. Sampling migrants

In cross national studies telephone samples are often used, partly to circumvent the aforementioned restrictions.

Random-digit-dialing (RDD) Commonly used to sample members of the general population for telephone

surveys; Requires (expensive) screening if specific subpopulations (such as migrants from

a specific country) are targeted.

Onomastic sampling (linguistic screening of names in telephone directories) Target population needs to be in the telephone directory; Risk of systematic undercoverage, e.g. individuals who:

▪ Have taken on another family name, for example due to marriage,▪ Live in shared housing, ▪ Engage in circular migration.

Only applicable in countries where comprehensive telephone registries exist problematic even within the EU (Pötzschke, Ciornei, & Apaydin vom Hau, 2014).

5

1. Sampling migrants

General research interest: (How) Could social networking sites (SNS) be used to sample migrants?

Why would it be a good idea?

▪ Size (e.g., in 2014: Facebook 1.39 billion active users per month

worldwide [Facebook, 2015b]; LinkedIn 347 million [LinkedIn, 2015]);

▪ Geographic coverage;

▪ (Potentially) Allows sampling of:

Individuals without telephone connection,

Small and scattered populations,

Very mobile populations.

6

1. Sampling migrants

Sampling of migrants in SNS – State of the art

▪ Use of existing groups (Baltar & Brunet, 2012; Crush et al., 2012; Oiarzabal, 2012; Ranieri, Manca, & Fini, 2012);

▪ Creation of specific groups by researchers (Brickman Bhutta, 2012);

▪ Use of snowball techniques (Baltar & Brunet, 2012; Brickman Bhutta, 2012; Crush et al., 2012).

7

1. Sampling migrants

Specific research questions:

a) Could Facebook advertisements be used to sample migrants into a study?

b) Could we reach individuals who would be missed by telephone sampling techniques?

Use of this approach in other areas:▪ Medical and health research

(Arcia, 2014; Chu & Snider, 2013; Fenner et al., 2012; Kapp, Peters, & Oliver, 2013; Lohse, 2013; Ramo & Prochaska, 2012);

▪ Recruiting participants for a cross-cultural study(Thomson and Ito 2014);

▪ Recruiting participants for cognitive pre-tests (Head, Dean, Flanigan, Swicegood, & Keating, 2016).

8

1. Sampling migrants

1. Sampling migrants (in cross-national research)

2. Study design

3. Results

4. Conclusions

9

Basic parameters▪ Online survey;▪ Comprehensive questionnaire (completion time approx. 14 min.)

Use of SNS and communication behavior, Migration and mobility experience, Demographics.

▪ Sampling via Facebook advertisements;▪ Targeting Polish migrants in Austria, Ireland,

Switzerland and the United Kingdom (aged 18 and above);

▪ Questionnaire could be reached via advertisement link or the project’s Facebook page;

▪ Questionnaire, ads and FB page in Polish only;▪ Field period: November 18, 2015 to January 13, 2016.

10

2. Study design

Settings of the Facebook ad campaign

▪ Campaign objective: “Send people to your website” Welcome page of our online survey;

▪ Targeting variables (phrasing according to options offered by

FB):

Behaviors: ‘expats (Poland)’ and ‘away from hometown’.

Location: Austria, Ireland, Switzerland & United Kingdom ‘people who live in this location’,

Age: ‘18-65+’,

Sex: ‘All’,

11

2. Study design

12

2. Study design

Fielding and costs of the campaign

▪ Cost-per-click-model;

▪ Campaign period: November 18 to December 17, 2015;

▪ Total Budget: 500 € (i.e., approx. 17 €/day);

▪ Ads were displayed on Desktop computers and mobile devices;

▪ Five advertisements:

All text components equal,

Only the used pictures differed.

13

2. Study design

14

2. Study design

© Dean Drobot/ shutterstock.com © Kalim/ Fotalia.com

© Contrastwerkstatt/ Fotalia.com © Production Perig/ Fotalia.com© Igor Mojzes/ Fotalia.com

Side note: Cambridge Analytica controversy

▪ Cambridge Analytica used data directly extracted from Facebook profiles;

▪ An app was employed to obtain these data, i.e. to scrape user profiles (Rosenberg et al. [New York Times], March 17, 2018).

▪ Key differences of the presented approach: Uses Facebook only as a sampling tool; Employs a regular service offered by Facebook to commercial

clients (i.e., advertisements); Researchers do not need – nor gain – direct access to data

contained in Facebook profiles; All analysed individual data are collected through a separate

survey participation is based on informed consent.

15

2. Study design

1. Sampling migrants (in cross-national research)

2. Study design

3. Results

4. Conclusions

16

17

3. Results

Estimated reach1

Unique users

reached

with ads1

Payed link

clicks1

Completed questionnaires2

N N N % Er % Uur % Plc

Austria 15,000 7,918 408 117 0.78 1.5 28.7

Ireland 54,000 28,107 1,314 425 0.79 1.5 32.3

Switzerland 9,000 3,432 215 62 0.69 1.8 28.8

UK 410,000 50,979 1,257 424 0.10 0.8 33.7

Source of absolute figures: 1 Facebook advertisement statistics; 2 Based on para data. Only respondents who reached the questionnaire via the Facebook advertisements; Relative values: own calculation.

▪ 1,103 completed questionnaires;▪ 1,100 with the advertisement period (i.e., first four weeks);▪ Survey costs per completed questionnaire: 0.45 €.

18

3. Results

Estimated reach1

Unique users

reached

with ads1

Payed link

clicks1

Completed questionnaires2

N N N % Er % Uur % Plc

Austria 15,000 7,918 408 117 0.78 1.5 28.7

Ireland 54,000 28,107 1,314 425 0.79 1.5 32.3

Switzerland 9,000 3,432 215 62 0.69 1.8 28.8

UK 410,000 50,979 1,257 424 0.10 0.8 33.7

Source of absolute figures: 1 Facebook advertisement statistics; 2 Based on para data. Only respondents who reached the questionnaire via the Facebook advertisements; Relative values: own calculation.

▪ 1,103 completed questionnaires;▪ 1,100 with the advertisement period (i.e., first four weeks);▪ Survey costs per completed questionnaire: 0.45 €.

Accuracy of targeting (I) - Country of Residence ▪ Targeting option: Living in Austria, Ireland, Switzerland or UK.

▪ Results:

1.082 respondents (98% of the sample) said they lived in one of the targeted countries,

Paradata are congruent with this answer for 1.040 respondents.

19

3. Results

CoR according to answer in questionnaireAustria Ireland Switzer-

land

UK Poland Other Total

Loca

tio

n

acco

rdin

g to

p

arad

ata

Austria 118 0 0 0 0 1 119

Ireland 0 436 1 1 1 3 442

Switzerland 0 1 65 0 0 0 66

UK 1 6 0 421 2 2 432

Poland 4 7 1 8 0 0 20

Other 3 2 5 2 0 12 18

Total 126 452 72 432 3 18 1,103

Accuracy of targeting (II) - Being a Polish migrant

▪ Targeting option: Expat Poland.

▪ Results:

Born in Poland, living abroad: 98.6 %,

Polish citizenship, living abroad: 97.7 %,

At least one of the above: 99.4 %.

20

3. Results

Accuracy of targeting (II) - Being a Polish migrant

▪ Targeting option: Expat Poland.

▪ Results:

Born in Poland, living abroad: 98.6 %,

Polish citizenship, living abroad: 97.7 %,

At least one of the above: 99.4 %.

Born in Poland & Polish citizenship, living abroad 97.0 %.

21

3. Results

Accuracy of targeting (III) - Age

▪ Targeting option: At least 18 years old.

▪ Results:

No (valid) response given by 23 respondents,

2 respondents younger than 18,

97.7 % of the sample in targeted age range.

Compliance with all targeting criteria

1.055 respondents = 96 % of the sample

22

3. Results

Reachability of respondents via telephone based sampling

▪ 97 % of the 1,000 migrant respondents used a country of residence (CoR) mobile number;

▪ Only 42% lived in a household which had a landline connection;

▪ Only 14% of the respondents were included in the CoR telephone book.

98% could, theoretically, be reached with expensiveRDD (mobile and landline numbers combined);

86% of the respondents were very unlikely to be reached with onomastic sampling.

23

3. Results

1. Sampling migrants (in cross-national research)

2. Study design

3. Results

4. Conclusions

24

Conclusions

▪ Facebook’s targeting mechanism works as advertised;

▪ Most respondents would probably not have been reached with onomastic procedure;

▪ Technique is cost efficient and not dependent on incentives;

▪ Existing research indicates that this technique can be used to sample other H2R populations;

▪ Suitable as part of a (cross-national) sampling strategy for hard-to-reach populations.

25

Thank you for your attention!

26

Contact:

Steffen Pötzschke (steffen.poetzschke@gesis.org)

For further details see:Pötzschke, Steffen and Michael Braun (2016): Migrant sampling using Facebook advertisements. A case study of Polish migrants in four European countries, Social Science Computer Review. doi: 10.1177/0894439316666262

LiteratureArcia, A. (2014). Facebook advertisements for inexpensive participant recruitment

among women in early pregnancy. Health Education & Behavior, 41(3), 237-241. doi:10.1177/1090198113504414

Baltar, F., & Brunet, I. (2012). Social research 2.0. Virtual snowball sampling method using Facebook. Internet Research, 22(1), 57-74. doi:10.1108/10662241211199960

Brickman Bhutta, C. (2012). Not by the book. Facebook as a sampling frame.Sociological Methods & Research, 41(1), 57-88. doi:10.1177/0049124112440795

Crush, J., Eberhardt, C., Caesar, M., Chikanda, A., Pendleton, W., & Hill, A. (2012). Diaspora on the web. New networks, new methodologies. In C. Vargas-Silva (Ed.), Handbook of research methods in migration (pp. 345-365). Cheltenham: Elgar.

Chu, J. L., & Snider, C. E. (2013). Use of a social networking web site for recruiting Canadian youth for medical research. Journal of Adolescent Health, 52(6), 792-794. doi:http://dx.doi.org/10.1016/j.jadohealth.2012.12.002

Dekker, R., & Engbersen, G. (2012). How social media transform migrant networks and facilitate migration. IMI Working Papers Series, No. 64, Retrieved April 9, 2014 from http://www.imi.ox.ac.uk/pdfs/imi-working-papers/WP-64-2012

EUROSTAT. (2016). Population on 1 January by five year age group, sex and citizenship. Retrieved April, 6, 2016, from http://ec.europa.eu/eurostat/en/web/products-datasets/-/MIGR_POP1CTZ

27

Facebook. (2014). A simpler way to organize and optimize campaigns on Facebook. Retrieved January, 1, 2016, from https://www.facebook.com/business/news/new-campaign-structure

Facebook. (2015a). Easy and effective Facebook adverts. Retrieved November, 17, 2015, from https://www.facebook.com/business/products/ads/

Facebook. (2015b). Form 10-K [Annual report 2014]. Retrieved April, 7, 2016, from http://files.shareholder.com/downloads/AMDA-NJ5DZ/1840407700x0x852173/F61276C5-0AE9-49DE-BFD9-087398F85EC8/FB2014AR.pdf

Fenner, Y., Garland, S. M., Moore, E. E., Jayasinghe, Y., Fletcher, A., Tabrizi, S. N., . . . Wark, J. D. (2012). Web-based recruiting for health research using a social networking site. An exploratory study. Journal of Medical Internet Research, 14(1) doi:10.2196/jmir.1978

Groenewold, G., & Lessard-Phillips, L. (2012). Research methodology. In M. Crul, J. Schneider & F. Lelie (Eds.), The European second generation compared. Does the integration context matter? (pp. 39-56). Amsterdam: Amsterdam Univ. Press.

Head, B. F., Dean, E., Flanigan, T., Swicegood, J., & Keating, M. D., (2016). Advertising for cognitive interviews: A comparison of Facebook, Craigslist, and snowball recruiting. Social Science Computer Review, 34(3), 360-377. Doi: 10.1177/0894439315578240

Kapp, J. M., Peters, C., & Oliver, D. P. (2013). Research recruitment using Facebook advertising. Big potential, big challenges. Journal of Cancer Education, 28(1), 134-137. doi:10.1007/s13187-012-0443-z

28

Komito, L. (2011). Social media and migration: Virtual community 2.0. Journal of the American Society for Information Science and Technology, 62(6), 1075-1086. doi:10.1002/asi.21517

LinkedIn (2015). Form 10-K [annual report 2014]. Retrieved April, 7, 2016, from https://s21.q4cdn.com/738564050/files/doc_financials/annual/2014/LinkedIn_Corporation_10-K_without_banners.PDF

Lohse, B. (2013). Facebook is an effective strategy to recruit low-income women to online nutrition education. Journal of Nutrition Education and Behavior, 45(1), 69-76. doi:10.1016/j.jneb.2012.06.006

Oiarzabal, P. J. (2012). Diaspora Basques and online social networks. An analysis of users of Basque institutional diaspora groups on Facebook. Journal of Ethnic and Migration Studies, 38(9), 1469-1485. doi:10.1080/1369183X.2012.698216

Pötzschke, S., Ciornei, I., & Apaydin vom Hau, F. (2014). Methodological report. In E. Recchi (Ed.), EUCROSS - final report (pp. 169-237). Retrieved July, 22, 2015 from http://nbn-resolving.de/urn:nbn:de:0168-ssoar-395269

Rosenberg, M., Confessore, N., & Cadwalladr, C. (2018). How Trump Consultants Exploited the Facebook Data of Millions. The New York Times, 2018/03/17. Retrieved April, 18, 2018 from https://nyti.ms/2GB9dK4

29

Ramo, D. E., & Prochaska, J. J. (2012). Broad reach and targeted recruitment using Facebook for an online survey of young adult substance use. Journal of Medical Internet Research, 14(1). doi:10.2196/jmir.1878

Ranieri, M., Manca, S., & Fini, A. (2012). Why (and how) do teachers engage in social networks? An exploratory study of professional use of Facebook and its implications for lifelong learning. British Journal of Educational Technology, 43(5), 754-769. doi:10.1111/j.1467-8535.2012.01356.x

Salentin, K. (2014). Sampling the ethnic minority population in Germany. The background to “Migration background”. Methods, Data, Analyses, 8(1), 25-52. doi:10.12758/mda.2014.002

Thomson, Robert and Naoya Ito (2014). Facebook Advertisements for Survey Participant Recruitment. Considerations from a Multi-National Study. International Journal of Electronic Commerce Studies 5 (2): 199-218. doi:10.7903/ijecs.1175.

30