sms data collection for measuring attitudes on migration

PowerPoint Presentation

Assessing the feasibility of using SMS Surveys in Eastern Europe to conduct a cross-country comparison of attitudes with regard to Europes migration crisisSara Gjoshevska, Sample SolutionsCarsten Broich, Sample Solutions3MC Conference 2016

Presentation OverviewIntroductionMethodologyMobile Phone SamplingQuestionnaire DesignData CollectionResults

7/30/20163MC Conference SMS Surveys 2

Why conduct SMS Surveys?Short Message Service meaning2 Way SMS Surveys Increasing mobile penetration rates Available for both smartphones and feature phonesFaster to collect dataCheaper (No budget for training, staffing, transportation, etc.)

7/30/20163MC Conference SMS Surveys 3

3

Methodology2-Way SMS survey is a self completion mode for Sensitive topics such as MigrationData Collection in 3 countries: Croatia, Macedonia and Poland10 questions 1st Q. opt-in and language choice2-7th Q. 1. Yes 2. No 3. Dont Know 4. Refuse to answer 8-10th Q. Demographic: Age, Gender, Postcode and EducationStratified Probability Sample

7/30/20163MC Conference SMS Surveys

4

2-7choice of answer on attitudeSMS 1: Would you like to participate in a short survey? Please answer with number: 1.Yes or reply STOP at any time for the survey to end. SMS 2: Does the refugee crisis in Europe frighten you? 1. Yes 2. No 3. Dont Know 4. Refuse to answer SMS 3: Do you believe that people with different ethnic or religious backgrounds should live in the same area or neighbourhood? 1. Yes 2. No 3. Dont Know 4. Refuse to answer SMS 4: In the past year have you or a family member ever felt threatened by the refugees? 1. Yes 2. No 3. Dont Know 4. Refuse to answerSMS 5: Do you agree with refugees being placed in the area or neighbourhood you live in? 1. Yes 2. o No 3. Dont Know 4. Refuse to answerSMS 6: Do you agree that refugees should be able to go to the same school with you or your family members? 1. Yes 2. No 3. Dont Know 4. Refuse to answerSMS 7: Who do you think is an authority on making decisions about asylum and migration? 1. The refugees 2. Government 3. Church 4. NGO 5. International Org. 6. Dont know 7. Refuse to answer

4

Mobile Phone SamplingMobile phone numbersStratified Probability Sample based on provider sharesNumbering blocks assigned to different providers7/30/20163MC Conference SMS Surveys

5

Random-digit dialling Sample (RDD)Nationally representative sampleMobile market share distributionStratification based on provider shares

5

Mobile Market Shares Distribution per Provider7/30/20163MC Conference SMS Surveys 6SIM penetration 112%

Questionnaire constructionDesign

Questionnaire design 10 Questions160 Characters no special characters (, , ,, , , )Unicode (70 characters) and pricing increaseTranslation and harmonization to Albanian, Croatian, Macedonian and Polish


7

Pretesting


8First Question Language selectionRouting based on LanguageImportance of Scales and ValuesInclusion of Dont know and RefuseSpeed of sending out SMSSMS Platform

FieldworkFieldwork completed in a few daysReminders after 24 hoursAchieving N=500 completes can be achieved in a cost-efficient manner within a few days in field. The questions asked should be simple Number of questions in the Survey7/30/20163MC Conference SMS Surveys

9

Achieving N=500 is not feasible in each country without prior knowledge and research of SMS usage and regulations in the marketThe number of questions in the survey has not showed significant impact on response rate.Reminders and thorough response coding are essential in achieving higher response rates.

9

Data Collection in Macedonia, Croatia and Poland7/30/20163MC Conference SMS Surveys

10CountryResponse Rate [%]Complete surveysSMS invitations SentCroatia1.2N=50642000Macedonia1.4N=52138000Poland0.25N=10242000

Target N=500 in reach countryPoland did not reach targetLow response rateOutcome of Poland does not yield comparable data.Used RR3 from AAPOR guidelines

Sampling ErrorNon-Coverage Provider restrictions in Croatia

Tele2 not represented in Sampling Frame

0.5% of the market share


11OperatorSMS Survey [%]Market Shares [%]VIP43.28%38.44T-Mobile56.71%60.98Tele20%0.5%

Underrepresented AreasRegional distribution is comparableLanguage limitations may cause error7/30/20163MC Conference SMS Surveys

12MacedoniaRegionSurvey[%]Census [%]Eastern11.718.57Northeastern4.618.51Pelagonia12.8611.19Polog2.5015.44Skopje42.8029.85Southeastern8.838.39Southwestern9.9810.64Vardar6.727.41

CroatiaRegionSurvey[%]Census [%]Central Croatia15.22%15.20%Eastern Croatia17.98%18%Southern Croatia21.74%21.80%Western Croatia8.30%8.30%Zagreb region36.76%36.80%

Non-response Under represented

In Macedonia the Albanian language is under covered

The survey is sent out in Macedonian, the population that does not speak Macedonian is excluded.

LanguageSMS Survey [%]Census [%]Macedonian66.49%89.44Albanian25.1%10.55

7/30/2016Sample Solutions Premier Sample Provider

13

Non Response Error Poland does not have strict data protection lawsSMS Spam in PolandIncentives - prize draw does not show increase in response rateSending of reminders to reduce non responseWaiting time to receive SMS


14

Pilot recommender for poland14

Response rates - Age & Gender Macedonia Age GroupsSMS Survey [%]Census [%]1524 years9.5916.382534 years29.5518.903544 years28.9817.604554 years 16.6916.725564 years 9.5916.2565+5.5614.13


15Croatia Age groups SMS Survey [%]Census [%]1524 years12.6516.072534 years21.7416.073544 years25.1015.744554 years 17.1917.295564 years 4.9416.0965+18.5820.88

Low response rates may lead to bias.

What we take away7/30/20163MC Conference SMS Surveys

16Design factors to support response ratePretesting, scales and weighting is important Dont Know importanceSpeed of sending out SMSRegional Surveys and similaritiesFuture study of tools to minimize errorFuture studies to test language routing options

SMS provides instant responses, use for instant data

Speed of sending out SMS- Importance of an efficient platform16

Thank you for listening!Contact:Sara Gjoshevska - [email protected] Broich - [email protected]

Sample Solutionswww.sample.solutions7/30/20163MC Conference SMS Surveys

17

sms data collection for measuring attitudes on migration

Business