our survey methodology in detail.docx

7/24/2019 Our Survey Methodology in Detail.docx

1/21

Our Survey Methodology in Detail

Sampling

The typical Pew Research Center for the People & the Press national survey selects a random

digit sample of both landline and cell phone numbers in all 50 U.. states and the !istrict of

Columbia. "s the proportion of "mericans who rely solely or mostly on cell phones for theirtelephone service continues to grow# sampling both landline and cell phone numbers helps to

ensure that our surveys represent all adults who have access to either $only about % of

households in the U.. do not have access to any phone'. (e sample landline and cell phonenumbers to yield a combined sample with appro)imately *5 of the interviews conducted by

landline and +5 by cell phone. This ratio is based on an analysis that attempts to balance cost

and fieldwor, considerations as well as to improve the overall demographic composition of the

sample $in terms of age# race-ethnicity and education'. This ratio also ensures an

adeuate number of cell only respondents in each survey.

The design of the landline sample ensures representation of both listed and unlisted numbers

$including those not yet listed' by using random digit dialing. This method uses random

generation of the last two digits of telephone numbers selected on the basis of the area code#telephone e)change# and ban, number. " ban, is defined as /00 contiguous telephone numbers#

for e)ample 0015551/%00 to 0015551/%22. The telephone e)changes are selected to be

proportionally stratified by county and by telephone e)change within the county. That is# thenumber of telephone numbers randomly sampled from within a given county is proportional to

that county3s share of telephone numbers in the U.. 4nly ban,s of telephone numbers

containing one or more listed residential numbers are selected.

The cell phone sample is drawn through systematic sampling from dedicated wireless ban,s of/00 contiguous numbers and shared service ban,s with no directory1listed landline numbers $to

ensure that the cell phone sample does not include ban,s that are also included in the landline

sample'. The sample is designed to be representative both geographically and by large and smallwireless carriers $also see cell phonesfor more information'.

oth the landline and cell samples are released for interviewing in replicates# which are small

random samples of each larger sample. Using replicates to control the release of telephone

numbers ensures that the complete call procedures are followed for all numbers dialed. The useof replicates also improves the overall representativeness of the survey by helping to ensure that

the regional distribution of numbers called is appropriate.

(hen interviewers reach someone on a landline phone# they randomly as, half the sample if they

could spea, with 6the youngest male# / years of age or older# who is now at home7 and theother half of the sample to spea, with the youngest female# / years of age or older# who is now

at home.7 8f there is no eligible person of the reuested gender currently at home# interviewers

as, to spea, with the youngest adult of the opposite gender# who is now at home. This method of
http://www.people-press.org/methodology/sampling/cell-phones/http://www.people-press.org/methodology/sampling/cell-phones/http://www.people-press.org/methodology/sampling/cell-phones/


2/21

selecting respondents within each household improves participation among young people who

are often more difficult to interview than older people because of their lifestyles.

Unli,e a landline phone# a cell phone is assumed in Pew Research polls to be a personal device.8nterviewers as, if the person who answers the cell phone is / years of age or older to

determine if the person is eligible to complete the survey $also seecell phone surveysfor moreinformation'. This means that# for those in the cell sample# no effort is made to give other

household members a chance to be interviewed. "lthough some people share cell phones# it isstill uncertain whether the benefits of sampling among the users of a shared cell phone outweigh

the disadvantages.

ampling error results from collecting data from some# rather than all# members of thepopulation. 9or each of our surveys# we report a margin of sampling error for the total sample

and usually for ,ey subgroups analy:ed in the report $e.g.# registered voters# !emocrats#

Republicans# etc.'. 9or e)ample# the sampling error for a typical Pew Research Center for the

People & the Press national survey of /#500 completed interviews is plus or minus %.2

percentage points with a 25 confidence interval. This means that in 25 out of every /00samples of the same si:e and type# the results we obtain would vary by no more than plus or

minus %.2 percentage points from the result we would get if we could interview every member ofthe population. Thus# the chances are very high $25 out of /00' that any sample we draw will be

within * points of the true population value. The sampling errors we report also ta,e into account

the effect of weighting. $"lso see (hy probability samplingfor more information'

Nonresponse

"t least ; attempts are made to complete an interview at every sampled telephone number. The

calls are staggered over times of day and days of the wee, $including at least one daytime call' to

ma)imi:e the chances of ma,ing contact with a potential respondent. 8nterviewing is also spreadas evenly as possible across the field period. "n effort is made to recontact most interview

brea,offs and refusals to attempt to convert them to completed interviews.

Response rates for Pew Research polls typically range from 5 to /5< these response rates arecomparable to those for other ma=or opinion polls. The response rate is the percentage of ,nown

or assumed residential households for which a completed interview was obtained. The response

rate we report is computed using the "merican "ssociation for Public 4pinion Research3s$""P4R' Response Rate * $RR*' method $9or a full discussion of response rates see ""P4R3s

tandard !efinitions'. 9ortunately# low response rates are not necessarily an indication of

nonresponse bias as we discuss in the problem of declining response rates.

8n addition to the response rate# we sometimes report the contact rate# cooperation rate# or thecompletion rate for a survey. The contact rate is the proportion of wor,ing numbers where a

reuest for an interview was made. The cooperation rate is the proportion of contacted numbers

where someone gave initial consent to be interviewed. The completion rate is the proportion ofinitially cooperating and eligible households where someone completed the interview.

Data weighting
http://www.people-press.org/methodology/methodology/collecting-survey-data/cell-phone-surveys/http://www.people-press.org/methodology/methodology/collecting-survey-data/cell-phone-surveys/http://www.people-press.org/methodology/sampling/why-probability-sampling/http://www.aapor.org/Standard_Definitions1.htmhttp://www.aapor.org/Standard_Definitions1.htmhttp://www.people-press.org/methodology/collecting-survey-data/the-problem-of-declining-response-rates/http://www.people-press.org/methodology/collecting-survey-data/the-problem-of-declining-response-rates/http://www.people-press.org/methodology/methodology/collecting-survey-data/cell-phone-surveys/http://www.people-press.org/methodology/sampling/why-probability-sampling/http://www.aapor.org/Standard_Definitions1.htmhttp://www.aapor.org/Standard_Definitions1.htmhttp://www.people-press.org/methodology/collecting-survey-data/the-problem-of-declining-response-rates/


3/21

>onresponse in telephone interview surveys can produce biases in survey1derived estimates.

urvey participation tends to vary for different subgroups of the population# and these subgroups

are li,ely to also vary on uestions of substantive interest. To compensate for these ,nownbiases# the sample data are weighted for analysis.

The landline sample is first weighted by household si:e to account for the fact that people inlarger households have a lower probability of being selected. 8n addition# the combined landline

and cell phone sample is weighted to account for the fact that respondents with both a landlineand cell phone have a greater probability of being included in the sample.

The sample is then weighted using population parameters from the U.. Census ureau for

adults / years of age or older. The population parameters used for weighting are? gender by age#gender by education# age by education# region# race and @ispanic origin that includes a brea, for

@ispanics based on whether they were born in the U.. or not# population density and among

non1@ispanic whites A age# education and region. The parameters for these variables are from the

Census ureau3s %0/% "merican Community uvey $e)cluding those in institutionali:ed group

uarters'# e)cept for the parameter for population density which is from the %0/0 Census. Thesepopulation parameters are compared with the sample characteristics to construct the weights. 8n

addition to the demographic parameters# the sample is also weighted to match current patterns oftelephone status $landline only# cell phone only or both landline and cell phone'# based on

e)trapolations from the %0/B >ational @ealth 8nterview urvey. The final weights are derived

using an iterative techniue that simultaneously balances the distributions of all weightingparameters. ou can view the demographic and phone usage uestions we use to compare the

sample characteristics to our weighting parametershere.

(eighting cannot eliminate every source of nonresponse bias. >onetheless# properly1conducted

public opinion polls have a good record in achieving unbiased samples. 8n particular# election

polling A where a comparison of the polls with the actual election results provides an opportunityto validate the survey results A has been very accurate over the years $see the >ational Council

on Public Polls Dvaluations of the %0/% and %0/0Dlections'.

Data analysis

Dach Pew Research survey report includes a 6topline uestionnaire7 with all of the uestions

from that survey with the e)act uestion wording and response options as they were read torespondents. This topline provides the results from the current survey for each uestion# as well

as results from previous surveys in which the same or similar uestions were as,ed.

9or discussion of the results in reports and commentaries# differences among groups are reportedwhen we have determined that the relationship is statistically significant and therefore is unli,elyto occur by chance. tatistical tests of significance ta,e into account the effect of weighting. 8n

addition# to support any causal relationships discussed# more advanced multivariate statistical

modeling techniues are often employed to test whether these connections e)ist# although theresults of these models may or may not be shown in the actual report.
http://www.people-press.org/files/2015/01/demographic_questions_2015.pdfhttp://www.people-press.org/files/2015/01/demographic_questions_2015.pdfhttp://www.people-press.org/files/2015/01/demographic_questions_2015.pdfhttp://www.ncpp.org/?q=node/137http://www.ncpp.org/?q=node/134http://www.people-press.org/files/2015/01/demographic_questions_2015.pdfhttp://www.ncpp.org/?q=node/137http://www.ncpp.org/?q=node/134


4/21

9or most studies# it is our policy to release datasets from Pew Research surveys five months after

the data was collected and archive them on our website as uic,ly as possible. Please visit our

datasetspage for further information.
http://www.people-press.org/category/datasets/http://www.people-press.org/category/datasets/


5/21


6/21


7/21


8/21


9/21


10/21


11/21


12/21


13/21


14/21


15/21


16/21


17/21


18/21


19/21


20/21


21/21

our survey methodology in detail.docx

Documents