the effects of adding a mobile- compatible design to the american life panel vera toepoel, utrecht...
TRANSCRIPT
The Effects of Adding a Mobile-Compatible Design
to the American Life Panel
Vera Toepoel, Utrecht University
Alerk Amin, RAND Corporation
Peter Lugtig, Utrecht University
Slide 2
RAND American Life Panel
• Nationally representative sample
• Internet surveys
• Provide computers / internet access to those who don’t have it
• Link responses to over 400 past surveys
• English / Spanish
Slide 3
ALP Summer 2014
• ~5000 respondents
• “Desktop Only” web interface for surveys
Slide 4
ALP Fall 2014
• Panel Refreshment
• Mid-Term Election Study
• Mobile-Compatible Interface
Slide 5
Panel Refreshment
• Recruit ~900 new respondents
• Recruitment with RDD– Asked to take a web survey– In web survey, asked about
interest/consent to join ALP– Added to panel and immediately
eligible to be selected for surveys
Slide 6
Mid-Term Election Study
• A series of surveys leading up to the November 2014 mid-term elections
• 6 weekly surveys– 1st survey opened September 28
2014– 6th survey opened November 5
2014
• Administered to all ALP panel members– Existing members– New RDD sample
Slide 7
Mobile-Compatible Interface
• New responsive design
• Compatible with desktops, phones, tablets
Slide 8
Original Interface - Desktop
Slide 9
Original Interface - Phone
Slide 10
New Interface - Desktop
Slide 11
New Interface - Phone
Slide 12
Introduction of Mobile Interface
Existing panel members sent notification about new interface with some screenshots
New RDD sample never saw the “desktop-only” interface
Slide 13
Research Questions
1. Has usage of mobiles and tablets changed since introduction of new design?
2. Are there differences in mobile adoption rates between existing panel members and new RDD recruits?
3. Which socio-demographics predict device use?
4. Is there a correlation between device use and voting behavior?
Slide 14
Analysis
• Track device usage during Mid-Term Election Study– User-Agent Strings to identify
device
Slide 15
SurveysSurvey Field Start Field End #
QuestionsNote
385 April 23 May 27 11 Last desktop-only survey for entire panel
406 September 28
October 04 9 1st mobile-compatible survey in ALP
407 October 05 October 11 6
408 October 12 October 18 6
409 October 19 October 25 6
410 October 26 November 01
6 Last survey before election
411 November 05
November 13
6 Post-election
Slide 18
Introduction of New Design –Existing Panel Members
PC Tablet Smartphone Other0
10
20
30
40
50
60
70
80
385406
Slide 19
Device Usage % PC Tablet Phone Other N
Wave 385
PM 73.4 10.2 15.2 1.2 2909
Wave 406
Total 71.8 11.1 16.0 1.1 1799
PM 68.8 10.0 19.4 1.7 1152
New ORC
77.1 13.0 9.9 0.0 647
Chi Square
42.18***
Wave 407
Total 73.2 10.2 15.7 .9 1755
PM 70.7 9.6 18.3 1.3 1112
New ORC
77.4 11.3 11.3 0.0 647
Chi Square
25.33***
Wave 408
Total 72.1 10.5 16.4 1.0 1755
Present PM
69.5 9.5 19.3 1.6 1101
New ORC
76.5 12.2 11.3 0.0 654
Chi Square
32.47***
Wave 409
Total 71.4 10.6 16.9 1.1 1893
Present PM
68.9 9.9 19.6 1.6 1164
New ORC
75.3 11.8 12.6 .03 729
Chi Square
24.46***
Wave 410
Total 72.1 9.4 17.3 1.2 1863
Present PM
69.6 9.4 19.2 1.8 1143
New ORC
76.0 9.6 14.3 .01 720
Chi Square
19.29***
Wave 411
Total 72.1 9.6 17.6 .8 1947
Present PM
70.3 8.6 20.0 1.2 1201
New ORC
74.9 11.1 13.7 .03 746
Chi Square
19.46***
Slide 20
Device Usage for 406
PC Tablet Phone Other0
10
20
30
40
50
60
70
80
ExistingNew RDD
Slide 21
Existing Panel Members Device Usage Over Time
385 406 407 408 409 410 4110%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
OtherPhoneTabletPC
Slide 22
Switching Devices Across Waves
Surveys
% Existing
% New RDD Sample
Total Chi Square
N
385-406
19.8 - 19.8 - 1073
406-407
10.6 9.9 10.3 .19 1513
407-408
12.2 9.4 11.1 2.82 1467
408-409
11.6 10.5 11.2 .45 1557
409-410
11.4 9.7 10.7 1.18 1666
410-411
10.9 12.2 11.4 .65 1685
Slide 23
Longitudinal Device Use –Existing vs New Panel Members
% Existing Panel Members
% New RDD Sample
Total
Always PC 60.8 67.4 63.3
Switch PC-tablet 6.7 7.3 6.9
Switch PC phone 15.3 12.4 14.2
Always tablet 5.1 6.6 5.7
Always phone 12.0 6.3 9.8
N 1187 742 1929Pearson Chi-Square 23.13, p=.000
Slide 25
Demographic Comparison – Existing vs New Panelmembers
% % Existing Panel Members
% New RDD Sample
Total Chi Square
N
BornUS 86.3 92.6 87.5 26.17*** 4543White 69.2 85.2 72.3 92.24*** Black 14.4 8.4 13.2 22.42*** Hispanic 28.1 4.8 23.5 215.99*** Male 38.1 48.3 40.1 30.97*** Separated 21.8 22.6 22.0 .26 Never Married
21.9 18.1 21.2 6.13**
Edu low 27.7 11.1 24.4 106.95*** Edu high 31.3 55.2 36.0 177.28*** Income low 18.5 7.6 16.4 62.06*** Income high
19.4 37.8 23.0 137.34***
hhsize>2 22.6 12.9 20.7 41.34*** work 58.6 55.8 58.0 2.32
Slide 26
Nominal Regression Device Use Over Six Waves With Demographics - Always PC Is Reference
B Switch PC-tablet
Switch PC-phone
Always tablet
Always phone
Intercept 1.78 .26 .23 2.24**
Age/10 -.01 -.54*** -.19** -.71***
Born US .38 -.01 .17 .41
White .22 .29 -.02 -.01
Black .83* .69* .79* 1.04***
Hispanic .71** .42*** .99*** .88***
Male -.40** -.18** -.45** -.17
Separated -.50* -.07 -.56** -.01
Never Married -.67* -.38* -.09 -.54*
Edu low -.36 -.09 .15 .14
Edu high .28 -.26* -.24 -.56**
Income low -.17 -.05 .25 .21
Income high .44** .22 .04 -.44
hhsize>2 .29 .07 .14 .34
work .17 .36* -.45* .06
New RDD Sample
.004 -.06 .61** .003
Slide 30
Voting Behavior in 2014 House Elections
Always PC
Switch PC-tablet
Switch PC-phone
Always tablet
Always phone
Republican
70.6 8.4 10.4 4.6 6.0
Democrat
73.4 7.0 11.2 4.3 4.0
Other 73.4 1.6 15.6 3.1 6.3No significant differences
Chi-square= 10.37, p=.22
Slide 31
Conclusions
• New design led some existing panel members to switch from PC to phones
• Tablet usage did not change
• Overall usage switched during introduction, then remained steady– About 10% “churn” each wave
• Device not correlated to voting behavior
Slide 32
Socio-Demographic ConclusionsMore Likely to Use Mobile• Young
• Black or Hispanic
• Working (switch PC-phone)
Less Likely to Use Mobile• Male
• Single
• Higher Education Level
• Working (tablet)
Slide 33
Thank You
Slide 34Los Angeles | London | New Delhi | Singapore | Washington DC
Vera Toepoel - Utrecht University
• October 2014
• £25.99 £20.75
• ISBN: 978-1-44624-967-3
Doing Surveys Online
This book helps students and researchers in identifying possible strategies to make the best use
of online surveys, providing pro’s and con’s, and do’s and don’ts for each strategy. It also explores the latest opportunities and developments that have
arisen in the field of online surveys, including using social networks, and provides expert guidance and
examples of best practice throughout.
Enter DISCOUNT CODE UK15CF31 at sagepub.co.uk or visit the SAGE stand to order
Offer valid till 31/12/2015 and cannot be used in conjunction with another offer.
20% DISCOUNT