Types of Methodologies

Surveys Part 2

Process

1. Determine the objectives2. Delimit the sample3. Construct the questionnaire4. Conduct the pilot study5. Establish the distribution process6. Analyze the data

4. Conduct the Pilot Study

1st pilot – colleagues, peers

2nd pilot – participants who are part of the intended population

What are we looking for?

4. Conduct the Pilot Study

Look for: Questions that lack discrimination

(answered the same by all) Unexpected responses

May be poorly worded Questions people don’t respond to Instructions Length of the survey How easy/difficult it is to analyze

Types of Surveys

1. Self-administered surveys

Mail Group

administration Internet E-mail

2. Interviews

Telephone Interview

Personal Interview Group Interview

Response Rate by Method

1. Web & mail 2. Web 3. E-mail 4. Mail

Mailed/Internet Survey Process

1. Advanced letter/e-mail 2. Survey 3. Reminder card/message 4. Final reminder card/message or

end survey

Internet Mail

Costs Software Mailing, printing

Data entry Automatic entry & coding

Time consuming entry & coding

Skill Need ability to develop survey

Same

Pre-test Need to test system & data entry

-

Missed items Higher than mail Lower than internet

Open ended ?’s

No difference No difference

Survey Structure & Analysis

Each question structure implies a “scale of measurement” Likert, semantic differential, etc.

Scales guide statistical analysis

Scales of Measurement

1. Nominal Scales

2. Ordinal Scales

3. Interval Scales

4. Ratio Scales

SOM- Nominal

Lowest level of measurement

Words & not # values….not a rating

Each value defines a distinct

category

Needs to be coded for Excel/SPSS

Example…..

_____ (1) Male _____ (2) Female

SOM - Nominal Data Other examples

2. What was your favorite activity? (pick

one)

___ Golf ___ Soccer ___Baseball

___ Golf (1) ___ Soccer(2) ___Baseball (3)

3. Have you attended this program

previously?

___Yes ___No

Unordered close-ended

SOM- Ordinal

Implied rank or order Each category has a position higher

than the previous Distance between categories isn’t

equal Ie. 1st, 2nd and 3rd place

Example AP Basketball poll

SOM - Ordinal

SOM - Interval

Ordered categories have meaningful size differences or distance between values

Distance between intervals is equal

Does not have a determined zero point Distance makes sense, ratios don’t

SOM - Interval

Calendar year 2011 ---365 days ---2012 ----365 days ---2013

Outdoor temperature Thermometer – degree increments are equal Is 0° as cold as it gets? 100 degrees isn’t twice as hot as 50

Meaningful size differences and equal distances (unlike ordinal)

Likert scales…

SOM - Interval

Likert scales Some believe ordinal

Categorical with order Others believe interval

Ordered categories have meaningful size differences

What is the difference between strongly agree, and agree….and is that difference the same as the difference between agree and neutral?

SOM - Interval

Likert scales Argument for interval

If there is symmetry in the responses SA/A/N/D/SD E/G/F/P – not symmetrical, uneven

distances between categories

SOM - Ratio

Has a true zero point Makes ratios possible

Has ordering & distancing properties of interval Weight

100 lbs. Is twice as heavy as 50 lbs. Reaction time Distance traveled: _____

What is your current age?

1. Under 182. 19-223. 23-264. 27 or older

Scale of measurement? Why?Would you rephrase the question? Why?How we ask the question may limit us

To summarize

Scales of measurement have range from the least precise to the most precise

Nominal Ordinal Interval Ratio

Why do we care? Determines what statistics are possible

Nominal is most limiting ; yes/no questions

56% enjoyed the show, 44% did not

Why change YES/NO questions to Likert? Degree of interest/opinions/attitudes With ordinal, interval, and ratio we can compute averages

and measure variability

Developing a Codebook

Before entering data into computer

Instructions on converting data

Details how to define & label each variable

Assign #’s (codes) to responses

List every variable…including the ID #

2. What is your current grad school sequence? ______ Athletic Training______ Biomechanics______ Exercise Physiology______ Physical Education Teacher Education______ Psychology of Sport & Physical Activity______ Recreation Administration______ Sport management

Developing a Codebook

Survey Quest

#

Variable name

Values Scale of measurement

Stats

2 Major 1= Athletic Training2=Biomechanics3=EP4=PETE5=PSP6=REC7=Sport mgt

Nominal Frequencies, %, mode

Developing a Codebook

Variable Name Use something you recognize & link to

the question Sales staff

Gender

Level of satisfaction or level of agreement is

likely not appropriate. SPECIFIC ENOUGH to

identify.

Developing a Codebook

Survey Quest

#

Variable name

Values Scale of measurement

Stats

2 Major 1= Athletic Training2=Biomechanics3=EP4=PETE5=PSP6=REC7=Sport mgt

Nominal Frequencies, %, mode

Developing a Codebook

Values 1=strongly agree

2=agree

3=disagree

4=strongly disagree

Values1=yes

2=no

1=male

2=female

Developing a Codebook

? # Variable name Values Scale of measurement

Stats

2 Learn of grad program

1= ISU website2=NCAA website3=Word of mouth4=Family & friends5=Other

Nominal Frequencies, %, mode

20 Trouble free transition

1=Strongly Agree2=Agree3=Neutral4=Disagree5=Strongly disagree

Interval Mean, median, mode

Other info….more challenging

“All that apply” question

Rank questions

Open ended questions

If this were the question? 1. How did you hear about this program? (check all that apply)

___Flyer ___Pantagraph

___ Word of mouth ___Brochure

___Radio ___Other (specify):______

1a Flyer 1=yes2=no

Nominal

1b Pantagraph 1=yes2=no

Nominal

1c Word of mouth 1=yes2=no

Nominal

1d Brochure 1=yes2=no

Nominal

1e Radio 1=yes2=no

Nominal

1f Other 1=yes2=no

Nominal

1g Other response n/a Nominal

Treat each option as a yes or no question (either checked/unchecked)?

If this were the question? 3. Rank each of the activities in order of preference (1=favorite, 3=least favorite)

_____Bingo

_____Yahtzee

_____Twister

Question #

Variable name

Values Scale Stats

3a Bingo 1=Favorite2=2nd favorite3=least favorite

Ordinal

3b Yahtzee 1=Favorite2=2nd favorite3=least favorite

Ordinal

3c Twister 1=Favorite2=2nd favorite3=least favorite

Ordinal

Not to be confused with….

What is the difference?

Question #

Variable name

Values Scale Stats

3 Favorite activity

1=Bingo2=Yahtzee3=Twister

Nominal

If this were the question? 3. Indicate your overall level of enjoyment at this event by circle the number…..

No Enjoyment Enjoyed Whatsoever 1 2 3 4 5 6 A Great deal

Semantic differentials example

Question #

Variable name

Values Scale Stats

3 Overall enjoyment

1=None whatsoever6=Enjoyed a great deal

Interval

No Enjoyment Enjoyed Whatsoever 1 2 3 4 5 6 A Great deal