Survey methods Survey methods 11

2

Methods of studyMethods of study

HistoryHistory Content analysisContent analysis Experimental designExperimental design Focus groupFocus group SurveySurvey

3

SurveySurvey

Is defined as a "means for Is defined as a "means for gathering information about the gathering information about the characteristics, actions, or characteristics, actions, or opinions of a large group of opinions of a large group of people." (AIO)people." (AIO)

Surveys can also be used to Surveys can also be used to assess needs, evaluate demand, assess needs, evaluate demand, and examine impact.and examine impact.

4

Steps in a proper studySteps in a proper study

A problem statement or a goalA problem statement or a goal Set a measurable objective for your Set a measurable objective for your

survey survey (See http://www.quorum-pr.com/Risorse/probjectives.pdf)(See http://www.quorum-pr.com/Risorse/probjectives.pdf)

Define the populationDefine the population Define the sampling choiceDefine the sampling choice Determine the method for Determine the method for

administeringadministering Develop an opening statement: the Develop an opening statement: the

question of anonymity versus question of anonymity versus confidentiality.confidentiality.

5

Steps in a proper study …Steps in a proper study …

Structuring the surveyStructuring the survey Institutional approvalInstitutional approval Pre-testPre-test CorrectionsCorrections Code sheet and software Code sheet and software

considerationsconsiderations Training the interviewersTraining the interviewers

6

Steps in a proper study …Steps in a proper study … Conducting the interviewsConducting the interviews Verification as necessaryVerification as necessary Data entry and debuggingData entry and debugging Initial data presentationInitial data presentation Interpreting the data, in contextInterpreting the data, in context

7

Types of data are related Types of data are related to statsto stats

Nominal: categorical, only by type.Nominal: categorical, only by type.

Ordinal: Rank order. Ordinal: Rank order.

Interval: Rating scale of 0 to 100, Interval: Rating scale of 0 to 100, arbitrary divisions.arbitrary divisions.

Ratio: Percentages and true zero.Ratio: Percentages and true zero.

8

Benchmark or?Benchmark or?

Benchmark: a moment in timeBenchmark: a moment in time

Longitudinal: at intervals over Longitudinal: at intervals over timetime

9

PopulationPopulation

The group that you are going to The group that you are going to study. study.

The population should be tightly The population should be tightly defined.defined.

Think “population frame.”Think “population frame.”

10

SampleSample

Non probability sample – a Non probability sample – a respondent is chosen at a place of respondent is chosen at a place of convenience.convenience.

Probability sample – a respondent Probability sample – a respondent has an equal opportunity of being has an equal opportunity of being in the survey. This sample is drawn in the survey. This sample is drawn typically from a list.typically from a list.

11

Convenience sampleConvenience sample

If you wanted to do a study of If you wanted to do a study of what shoppers thought about what shoppers thought about Global Village, you would choose Global Village, you would choose folks shopping at Global Village.folks shopping at Global Village.

12

Probability sampleProbability sample

Everyone has an equal chance of Everyone has an equal chance of being chosen as a respondent.being chosen as a respondent.

Folks are chosen as an Nth sample Folks are chosen as an Nth sample from a complete list.from a complete list.

Folks could be chosen Folks could be chosen proportionately from a list, say proportionately from a list, say Freshman, Sophomores, Juniors, Freshman, Sophomores, Juniors, and Seniors.and Seniors.

13

Nth sampleNth sample

400 respondents from a population of 400 respondents from a population of 4,0004,000

Divide 400 into 4,000. The answer is 10.Divide 400 into 4,000. The answer is 10.

Pick a number at random between 1 and Pick a number at random between 1 and 10.10.See http://www.programurl.com/software/random-number-generator.htm.See http://www.programurl.com/software/random-number-generator.htm.

Say that number is 7. The first respondent Say that number is 7. The first respondent is 7, the second is 17, the third is 27 and is 7, the second is 17, the third is 27 and so on.so on.

14

Sample sizeSample size

Sample size depends upon time, Sample size depends upon time, money, and desired error rate.money, and desired error rate.

The error rate is 1/square root of The error rate is 1/square root of the sample size.the sample size.

15

Sample error rateSample error rate

A sample of 400 produces an A sample of 400 produces an error rate of error rate of

1/square root of 400 is 20.1/square root of 400 is 20.

1 divided by 10 is .051 divided by 10 is .05

The error rate is + or – 5 percent.The error rate is + or – 5 percent.

16

Sample error rateSample error rate A sample of 1,600 would produce an A sample of 1,600 would produce an

error rate of + or – 2.5%.error rate of + or – 2.5%.

The square root of 1,600 is 40.The square root of 1,600 is 40.

The error rate is 1/40. The error rate is 1/40.

The error rate is + or – 2.5%.The error rate is + or – 2.5%.Confused see, http://www.robertniles.com/stats/mean.shtml.Confused see, http://www.robertniles.com/stats/mean.shtml.

17

Population sizes and Population sizes and sample sizesample size

PopulationPopulation +/- 3%+/- 3%

200200 100100 300300 150150 400400 200200 500500 250250

If you want the table for this, see Frey, L.R., Botan, C., & Kreps, If you want the table for this, see Frey, L.R., Botan, C., & Kreps, G.L. (1999). G.L. (1999). Investigating CommunicationInvestigating Communication. You need this list for . You need this list for populations to about 1,000.populations to about 1,000.

18

Cooperation rateCooperation rate

This is the percentage of those This is the percentage of those who agree to participate.who agree to participate.

The cooperation rate varies from The cooperation rate varies from 75% to 25% depending upon the 75% to 25% depending upon the method used, survey fatigue, and method used, survey fatigue, and how much money that you have.how much money that you have.

This means you might have to This means you might have to choose more.choose more.

19

Type of question formatType of question format

TunnelTunnel: similarly organized questions: similarly organized questions

FunnelFunnel: begins with broad open : begins with broad open questions followed by narrower questions followed by narrower closedclosed questions questions

InvertedInverted funnel: begins with narrow, funnel: begins with narrow, closed questions and builds to closed questions and builds to broader, broader, open questions.open questions.

20

Arranging the formatArranging the format

Treat all questions as a whole.Treat all questions as a whole.

Group questions logically.Group questions logically.

Place easy to answer questions at Place easy to answer questions at the end because of respondent the end because of respondent fatigue.fatigue.

21

Arranging the formatArranging the format

Place sensitive questions in the Place sensitive questions in the middle, middle, afterafter the respondent is the respondent is comfortable and comfortable and beforebefore she is she is tired.tired.

Six or more similar items cause Six or more similar items cause fatigue. Vary what you ask.fatigue. Vary what you ask.

22

WordingWording

Use complete sentences.Use complete sentences.

Write questions in conversation Write questions in conversation tones.tones.

UnderlineUnderline, , italicizeitalicize, or , or bold bold important items.important items.

23

WordingWording

Ask about a limited time period, Ask about a limited time period, specify specify it. Yesterday, week, month, it. Yesterday, week, month, but don’t make it a memory test.but don’t make it a memory test.

Use Use otherother, but sparingly., but sparingly.

Be careful with letting someone Be careful with letting someone answer answer don’t know.don’t know.

24

Wording…Wording…

Use standard demographic Use standard demographic questions.questions.

Avoid all inclusive terms such as Avoid all inclusive terms such as alwaysalways or or nevernever..

25

Wording …Wording …

If you use opened-ended questions If you use opened-ended questions ask the ask the positive ones firstpositive ones first, the , the negative ones secondnegative ones second. (What do you . (What do you like best, what do you like least.)like best, what do you like least.)

Don’t embarrass. Ask “What is the Don’t embarrass. Ask “What is the highest school level you highest school level you completed,” rather than “Did you completed,” rather than “Did you graduate from College?”graduate from College?”

26

Good survey questions Good survey questions areare

StraightforwardStraightforward

Ask about only one issue (if it Ask about only one issue (if it has an has an andand break it up) break it up)

Don’t load questions. “Don’t ask, Don’t load questions. “Don’t ask, Don’t you think that…”Don’t you think that…”

27

Demographic blockDemographic block

AgeAge Educational levelEducational level GenderGender OccupationOccupation Place of residencePlace of residence

28

Educational categoriesEducational categories

Does not have any formal Does not have any formal educationeducation

Some elementary educationSome elementary education Finished elementary educationFinished elementary education Finished secondary, did not Finished secondary, did not

attend university or collegeattend university or college Graduated from collegeGraduated from college Some post-graduate educationSome post-graduate education Post-graduate educationPost-graduate education

29

Categorical questionsCategorical questions

Should have equal groupingsShould have equal groupings What is your age?What is your age?

Less than 10Less than 10 10-1910-19 20-2920-29 30-3930-39 40-4940-49 Or see, http://factfinder.census.gov/servlet/QTTable?_bm=y&-Or see, http://factfinder.census.gov/servlet/QTTable?_bm=y&-

geo_id=01000US&-qr_name=DEC_2000_SF1_U_QTP12&-geo_id=01000US&-qr_name=DEC_2000_SF1_U_QTP12&-ds_name=DEC_2000_SF1_U.ds_name=DEC_2000_SF1_U.

30

Cultural concernsCultural concerns

Symbolic words are abstract. They Symbolic words are abstract. They do not mean the same time to all do not mean the same time to all peoplepeople

Abstract: love, honor, be safe, Abstract: love, honor, be safe, dignitydignity

Concrete: table, chair, love, Concrete: table, chair, love, househouse??

31

SymbolSymbol

I love you.I love you. I love broccoli.I love broccoli. I love this game.I love this game.

32

Semantic rulesSemantic rules I will call you soon.I will call you soon.

Value Value good, badgood, bad PotencyPotency strong, weakstrong, weak ActivityActivity fast, slowfast, slow

Low-context cultureLow-context culturegood, strong, fastgood, strong, fast High-context cultureHigh-context culture good, weak, good, weak,

slowslow

33

Likert-scalesLikert-scales

Gauge the degree to which there Gauge the degree to which there is agreement or disagreementis agreement or disagreement

5, 7, 9, 11 point scales, 5, 7, 9, 11 point scales, depending upon sensitivity and depending upon sensitivity and number of respondentsnumber of respondents

34

Likert scale sampleLikert scale sample

Circle one answer for each Circle one answer for each questionquestion

I think research is excitingI think research is exciting

SASA AA N or DNAN or DNA DD SDSD

But consider “forced choice.”But consider “forced choice.”

35

Or, rephrase the Or, rephrase the questionquestion

00 5050 100100 150150

Respondents report a number to Respondents report a number to

rate something.rate something.

36

Thurstone scales (rocket Thurstone scales (rocket science)science)

Place a check mark in the blank Place a check mark in the blank next to each statement with which next to each statement with which you agree.you agree.

__ This class is more challenging __ This class is more challenging than the other course I am taking.than the other course I am taking.

__This class teaches me valuable __This class teaches me valuable information.information.

__This class is what I expected.__This class is what I expected. __This class is interesting.__This class is interesting.

37

Semantic differential Semantic differential scalesscales

Rate the concept, “the president” Rate the concept, “the president” according to the way you feel according to the way you feel about it by placing an X on each about it by placing an X on each of the seven point scales to of the seven point scales to indicate your evaluation.indicate your evaluation.

38

The scaleThe scale

The presidentThe president

Reliable _:__: __:__:__:__:__Unreliable*Reliable _:__: __:__:__:__:__Unreliable*Uninformed _:__: __:__:__:__:__InformedUninformed _:__: __:__:__:__:__InformedUnqualified _:__: __:__:__:__:__QualifiedUnqualified _:__: __:__:__:__:__Qualified

Dishonest _:__: __:__:__:__:__HonestDishonest _:__: __:__:__:__:__HonestIntelligent _:__: Intelligent _:__:

__:__:__:__:____:__:__:__:__Unintelligent*

* Indicates reverse scoring

39

Guttmann scaleGuttmann scale

Consider a person who is Chinese, Consider a person who is Chinese, and circle the number next to and circle the number next to each of the situations listed each of the situations listed

below in which you would accept below in which you would accept a person from this group.a person from this group.

40

The scaleThe scale

1.1. As a visitor to my countryAs a visitor to my country2.2. To citizenship in my countryTo citizenship in my country3.3. To employment in my occupationTo employment in my occupation4.4. To my street as my neighborTo my street as my neighbor5.5. To my club as a personal friendTo my club as a personal friend6.6. To close kinship by marriageTo close kinship by marriage

Bogardus Social Distance ScaleBogardus Social Distance Scale

41

If you are stuck, look atIf you are stuck, look at

Rubin, R.B., Palmgreen, P. & Rubin, R.B., Palmgreen, P. & Sypher, H.E. (2004). Sypher, H.E. (2004). Communication Research Communication Research Measures: A SourcebookMeasures: A Sourcebook. . Mahwah, New Jersey: Lawrence Mahwah, New Jersey: Lawrence Erlbaum Associations.Erlbaum Associations.

42

Presenting the dataPresenting the data

Asking the questions so that you Asking the questions so that you get what you want.get what you want.

Statistics for People who (Think Statistics for People who (Think They) Hate StatisticsThey) Hate Statistics