chapter 10 questionnaire surveys: typology, design and coding

Chapter 10 Questionnaire

surveys: typology, design and coding

CONTENTS

• Introduction• The household questionnaire survey• The street survey• The telephone survey• The mail survey • E-surveys• User/site/visitor surveys• Captive group surveys• Questionnaire design• Coding• Validity of questionnaire-based data• Conducting questionnaire surveys

Introduction

• Definitions and terminology• Roles• Merits of questionnaire surveys• Limitations• Interviewer-completion or respondent-completion?• Types of questionnaire survey

A. J. Veal & S. Darcy (2014) Research Methods for Sport Studies and Sport Management: A practical guide. London: Routledge

Definitions

• Questionnaire or ‘interview schedule’:– A printed or on-line list of questions

• Survey– Whole process of conducting an investigation which

involves a number of ‘subjects’

• Questionnaire survey– A survey involving the use of a questionnaire

• ie. a ‘survey’ is not a ‘questionnaire’


Roles of questionnaire surveys

• Used when a specified range of information required• Typically involve just a sample of the population

being studied – for implications see Ch. 13, Sampling

• But, the aim is to make inferred statements about the population as a whole


Merits of questionnaire surveys

• An ideal method of providing policy-related data• Transparent methodology• Quantification easily communicated/understood• Repeat surveys can study change over time• Can cover a wide range of (sporting) activities• Can study attitudes, meanings, perceptions of

population as a whole


Limitations of questionnaire surveys

• Samples – see Ch. 13• Self-reported data

– exaggeration/under-reporting– accuracy of recall– sensitivity to some questions


Interviewer-completion or respondent-completion?

Interviewer-completion Respondent-completion

Advantages - More accuracy- Higher response rates- Fuller & more complete

answers- Design can be less 'user-

friendly'

- Cheaper- Quicker- Relatively anonymous

Disadvantages - Higher cost- Less anonymity

- Patchy response- Incomplete response-Risk of frivolous responses- More care needed in design


Types of questionnaire survey (Fig. 10.3)

Type Int. or resp.completion

Cost Sample drawn from:

Response rate

Household Standard Either High Whole popn* High Time-use Resp. High Whole popn* High Omnibus Either Medium Whole popn* High

* refers to population of subjects to be studied – see ch. 13


Types of questionnaire survey contd (Fig. 10.3)


Cost Sample drawn from

Response rate

Street Interviewer Medium Most of pop’n MediumTelephone Interviewer Medium People with

land-line phoneHigh but falling

Mail Respondent Cheap General or Special

Low


Types of questionnaire survey contd (Fig. 10.3)


Cost Sample drawn from

Response rate

E-survey Respondent Cheap People access-ible via email/ internet

Medium

On-site Either Medium Site users only High Captive Respondent Cheap Captive group

onlyHigh


Household survey

• Nature – Can cover all adult age-groups– Representative of whole community – Common for government, market research, etc.

• Conduct– Typically interviewer-completed, but also ‘drop-off and collect’ with

respondent-completion sometimes used – or combination– Sampling – see chapter 13

• Omnibus surveys– One questionnaire includes questions on a number of topics for

multiple clients• Time-budget studies

– Respondents complete a 1 or 2-day diary of activities• National surveys

– See Chapter 7: often seen as secondary data sources


Street surveys

• Nature– Conducted in:

• Shopping street/mall• Tourism areas (but may be seen as ‘site survey’)• Transport nodes (bus-stations, airports) - tourism

• Conduct– Interviewers typically given quotas related to known

demographics of the community– Still a problems of representativeness re people who

• are housebound • do not visit shopping streets • do not visit particular tourist locations (eg. VFR sport tourists)


Telephone survey

• Nature– Common for political polling– Growing response and representativeness problem

because of: • excluding those without land-line telephones• growing resistance/non-response in some countries

• Conduct– Fast, using CATI – computer-aided telephone interviewing

• Numbers selected/dialed automatically• Data keyed directly into computer

– Not possible to show checklists to respondents– Anonymity may help honesty of response


Mail/postal survey

• Nature– Ideal for surveying widely dispersed lists

• eg. Members of organisations

• Conduct– Low response rates – eg. 30% often quoted as ‘acceptable’

…. but is it?– Factors affecting response rates – see below

• Mail and user/site survey combos– In some on-site user/visitor surveys a brief face-to-face

interview is combined with a hand-out questionnaire which respondents complete and mail back after their visit.


Factors affecting mail survey response rates (Fig. 10.4)

1. Interest of the respondent in the survey topic2. Length of the questionnaire3. Questionnaire design/presentation/complexity4. Style, content, authorship of accompanying letter5. Provision of a postage-paid reply envelope6. Rewards for responding7. Number and timing of reminders/follow-ups – see

below


Mail survey response pattern (Fig. 10.6)

0

10

20

30

40

50

60

70

2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50Days

ReminderCard sent Follow-up

Letter sent

Start

0

10

20

30

40

50

60

70

80

Daily returns

Cumulative%


E-surveys

• Nature– Conducted via email/Internet– Dependent on availability of email list or other means of

inviting sample to participate• Conduct

– Partially or fully electronic– Commercial organisations offer online service, including

on-line questionnaire design, and analysis. – Problem of response rate due to growth of ‘junk mail’


Types of e-survey (Fig. 10.7)

Type Request Questionnaire Completion ReturnHybrid email/mail Email Attached text file Manual on hard

copy Mail

Hybrid email Email Attached text file Word-process-or/spreadsheet

Email + text file

Fully electronic: ad hoc

Email On-line, interactive Online On-line submission

Fully electronic: panel

Panel email

On-line, interactive Online On-line submission


User/on-site/visitor surveys• Nature

– Survey conducted at a leisure facility/site– Alternative terms:

• On-site survey, site survey, user survey, visitor survey, spectator survey

– Commonly used by facility/site managers• Conduct

– Interviewer completion preferred for quality/ response rate– Respondent-completion, if not very closely supervised, can result in:

• Low response rates• Unrepresentative sample• Poor quality responses (incomplete etc.)

• On-site/mail combo– face-to-face interview can be followed by handout of a mail-back

questionnaire – particularly in long-stay sites, such as cricket, golf match, motor sport


Uses of User/on-site/visitor surveys

• Establish catchment area (place of residence)• User profile (age, gender, socio-economic group etc.)• User opinions• Non-users :

– Use census data to assess non-users within the catchment area


Captive group surveys

• Nature– Respondents in organised group– May have little choice but to participate– Ethically, must be given the option

• Conduct– Typically respondent-completed under supervision– Quick and cheap


Questionnaire design: Contents

• Introduction: research problems and information requirements

• Example questionnaires• General design issues• Types of information• Activity/events/places questions• Respondent characteristics• Attitude/opinion questions• Market segments• Ordering of questions and layout of questionnaires


Questionnaire design: culmination of a process (Fig. 10.8)

Research questions/ management problem/ conceptual framework

List of information requirements

Research strategy

Questionnaire survey

Pilot/ test

Other methods?

Final design

Draft design


Example questionnaires: Case study 10.1

A. Questionnaire for a site/street survey: respondent-completed: Campus Sporting Life Survey

B. Questionnaire for a household survey: interviewer-completed: Short-stay Sport Holiday Survey

C. Questionnaire for a site survey: interviewer-completed: Ramsey Street Park Survey


General design issues: wording of questions (Fig. 10.9)

Principle Bad example Improved versionUse simple language

What is your frequency of utilisation of sport goods retailers?

How often do you use sports shops?






Avoid ambiguity Do you play sport very often?

Have played any of the following sports within the last four weeks? (show list)








Avoid leading questions

Are you against the extension of the airport?

What is your opinion on the extension of the airport? Are you for it, against it or not concerned?








Avoid leading questions

Are you against the extension of the airport?

What is your opinion on the extension of the airport? Are you for it, against it or not concerned?

Ask just one question at a time.

Do you use the local sports centre, and if so what do you think of its facilities?

1. Do you use the local sports centre? Yes/No2. What do you think of the facilities in the local sports centre?


Pre-coded vs open-ended questions (Fig. 10.10)

• Open-ended:– What is the main constraint on your ability to study?

________________________________________• Pre-coded/closed:

– A. My job ☐1

– B. Timetabling ☐2

– C. Child care ☐3

– D. Spouse/partner ☐4

– E. Money ☐5

– F. Energy ☐6

– G. Other ____________ ☐7

• In interview situation card shown to intervieweeA. J. Veal & S. Darcy (2014) Research Methods for Sport Studies and Sport Management: A practical guide. London: Routledge

Open-ended questions produce large numbers of answers (Fig. 10.11), which will require coding (see later)

• Question: Do you have any complaints about this (beach/picnic) area? (Site survey in a beachside National Park with boating and camping. Number of responses in brackets)

– Sand bars (22) - Uncontrolled boats (23)– Parking (5) - Jet skis (39)– Wild car driving (1) - Surveys (1)– Lack of beach area (1) - Should be kept for locals (1)– Too few shops (1) - Seaweed (3)– Too few picnic tables (4) - Need showers (1)– No timber for barbecue (2) - Administration of National Park (1)– Need more picnic space (3) - Maintenance & policing of Park (1)– Need boat hire facilities (1) - Trucks on beach (2)– Need active recn facilities (1) - Anglers (1)– Litter/pollution (74) - Crowds/tourists (26)– Urban sprawl (1) - Having to pay entry fee (6)– Need wharf fishing access (1) - Houses along waterfront (2)– Lack of info. on walking trails (1) - Unpleasant smell (drain) (2)– Not enough facilities (3) - Sales people (1)– Slow barbecues (2) - Need electric barbecues (1)– Etc. - Etc


Types of information

1. Activities/events/places What?2. Respondent characteristics Who?3. Attitudes/motivations Why?


Activity/event/places questions

• Effect of participation reference period• See Table 10.1


Participation in sport etc. by reference period, Britain, 1987: Table 10.1 (part)

% of persons aged 16+ participating in last:

A. year B. 4 weeks Ratio of A:BWalking 68.2 44.5 1.5Swimming: indoors 35.1 12.8 2.7Swimming: outdoor 14.9 2.9 5.1Keep fit/yoga 20.7 12.3 1.7Skiing 2.6 0.3 8.7


Respondent characteristics: range of data

• Gender• Age• Economic status• Occupation/social class (own or 'head of household')• Previous job history• Income (own or household)• Education/qualifications

• Marital/family status • Household type/family size• Life-cycle• Ethnic group/country of

birth• Residential location• Mobility - driving licence,

access to private transport• Party/group size/type

(site/visitor surveys)


Attitude/opinion questions

• Formatsa. Open-ended or direct questionsb. Checklistc. Ranking d. Likert scalese. Attitude statementsf. Semantic differentialg. Repertory grid


Attitude/opinion questions contd (Fig. 10.17)

a. Open-ended/direct: What attracted you to apply for this course?____________________________________

b. Checklist: Of the items on the card, which was the most important to you in applying for this course?A. Good reputationB. Easy accessC. CurriculumD. Level of feesE. Easy parking



c. Ranking: Please rank the items on the card in terms of their importance to you in choosing a course. Please rank them 1 for the most important to 5 for the least important.

RankA. Good reputation ___B. Easy access ___C. Curriculum ___D. Level of fees ___E. Easy parking ___



d. Likert scales: Looking at the items on the card, please say how important each was to you in choosing this course; was it: Very important, Quite important, Not very important or Not at all important?

Very Quite Not very Not at allimportant important important important

Good reputation □1 □2 □3 □4

Easy Access □1 □2 □3 □4

Curriculum □1 □2 □3 □4

Level of fees □1 □2 □3 □4

Easy parking □1 □2 □3 □4



e. Attitude Statements: Please read the statements below and indicate your level of agreement or disagreement with them by ticking the appropriate box.

Agree Agree No Disagree DisagreeStrongly opinion strongly

The learning experience is more important than the

qualification in education □1 □2 □3 □4 □5

Graduate course fees are

too high □1 □2 □3 □4 □5



f. Semantic differential: Please look at the list below and tick the line to indicate where you think this course falls in relation to each factor listed.

Difficult |_______|________|________|________| EasyIrrelevant |_______|________|________|________| RelevantProfessional |_______|________|________|________| UnprofessionalDull |_______|________|________|________| Interesting


Introductory remarks, Ordering of questions

• Consider content/length of opening/introductory remarks

• Question order:– Start with easy questions– Start with 'relevant' questions– Leave sensitive questions until later

• Layout:• Be aware of the needs of the reader/user – interviewer or

respondent?• Special care with mail survey questionnaires• Compactness (eg. single page) = ease of handling• Two-column layout often helps


Coding

• Giving numerical codes to answers:– eg. Yes: 1 No: 0– alpha codes can be used, but numeric has

advantages• Coding of open-ended questions: Fig. 10.19


Answers from 25 respondents to the question: 'What suggestions would you make for improving campus sporting life?‘ (Fig. 10.19)More sport available ///Upgrade gym facilities ///More weekday events //More lunch-time events /More evening events //Better music in gym /Cheaper drinks ///Free transport from city /

Better food ///Keep out non-students //Better spectator accommodation ///Better coaches/instructors //More classes for women //Lower membership costs //Warmer pool water //Better control of facility users //

Suggested coding system code Comments on programme content 1 Comments on timing 2 Comments on facilities 3 Comments on costs 4 Comments on organisation 5 Other 6


Recording coded information (Fig. 10.20)

Campus Sporting Life Survey 2008| Office Use

| # 1 | qno

1. Which of the following best describes your current situation? |Full-time student with no regular paid work 1 |Full-time student with some regular paid work 2 | 2 statusPart-time student with full-time job 3 | Part-time student - other 4 |

ONLY ONE ANSWER POSSIBLE – ONE CODE – ONE VARIABLE (status)



6. What suggestions would you make for improving campus social life? ||

Provide more flunchtime sessions___ _________________ | 1 sug1 ________________________________________________ | __ sug2 ________________________________________ | __ sug3

|OPEN-ENDED (CODING SEE Fig. 10.19) – UP TO THREE ANSWERS |RECORDED = THREE VARIABLES |

|7. You are: Male 1 Female 2 | 2 gender

|8. Your age last birthday was: 22 years | 22 age


Data from first completed questionnaire (Fig. 10.21)

qno status cafebar music sport travel cheap daytme unusual meet quality spend relax social mental sug1 sug2 sug3 gend age

1 2 1 1 0 0 1 4 2 3 5 100 3 3 1 1 1 18

Variables

Data from one questionnaire (as in Fig 10.20)


Data from 15 completed questionnaires (Fig. 10.21)

qno status cafebar music sport travel cheap daytme unusual meet quality spend relax social mental sug1 sug2 sug3 gend age

1 2 1 1 0 0 1 4 2 3 5 100 3 3 1 1 1 18

2 2 1 1 1 0 1 4 2 3 5 50 2 3 1 2 1 1 19

3 3 1 0 0 0 2 5 1 3 4 250 2 2 2 3 4 2 19

4 4 0 0 0 0 2 3 1 4 5 25 3 2 2 1 2 4 1 22

5 3 1 0 0 1 1 4 3 2 5 55 3 3 1 2 24

6 3 1 1 1 0 2 4 1 3 5 40 2 3 1 2 2 20

7 2 1 0 0 0 3 2 1 4 5 150 2 3 2 3 2 20

8 2 1 0 1 0 3 4 2 1 5 250 1 2 2 4 5 1 21

9 4 0 1 0 0 1 5 2 3 4 300 2 3 2 1 21

10 3 1 1 0 0 2 3 1 5 4 100 1 2 1 1 1 2 21

11 3 1 1 0 1 2 3 1 4 5 75 2 2 1 2 3 2 19

12 2 1 0 1 0 1 4 3 2 5 50 2 3 1 1 22

13 1 1 0 1 0 1 5 2 3 4 55 2 3 2 1 2 2 21

14 3 1 1 0 0 2 4 1 3 5 75 3 3 2 4 2 20

15 1 1 1 0 0 3 2 1 5 4 150 3 3 1 1 2 5 1 20

Continued in Chapter 16A. J. Veal & S. Darcy (2014) Research Methods for Sport Studies and Sport Management: A practical guide. London: Routledge

Validity/ reliability of questionnaire-based data (Fig. 10.22)

• General:• Non-response• Questionnaire design: lack of clarity• Accuracy of recall• Desire to impress• Privacy concerns/sensitivity• Language/accent• Interviewee patience/fatigue• Physical context• Interviewer-administered• Interviewer-respondent rapport• Interviewer consistency• Respondent-completed• Literacy• Non-completionA. J. Veal & S. Darcy (2014) Research Methods for Sport Studies and Sport Management: A practical guide. London: Routledge

Checking validity

• Dummy questions or answer categories• Semi-disguised duplication of questions• Comparing participation time periods• Use of/comparing alternative data sources


Taking account of validity problems?

• Not much evidence in the literature


Conducting questionnaire surveys

• Planning fieldwork: tasks (Fig. 10.23)a. Seek permissions - to visit sites, obtain records, etc. b. Obtain lists for sampling – eg. voters listsc. Arrange printing – of questionnaires etc. d. Check insurance issuese. Prepare written instructions for interviewersf. Prepare identity badges/letters for interviewersg. Recruit interviewers and supervisorsh. Train interviewers and supervisors i. Obtain quotations for any fieldwork to be conducted by

other organisations j. Appoint and train data coders/processorA. J. Veal & S. Darcy (2014) Research Methods for Sport Studies and Sport Management: A practical guide. London: Routledge

Conducting a pilot survey

• Pilot survey purposes (Fig. 10.24)

a. Test questionnaire wordingb. Test question sequencingc. Test questionnaire layoutd. Code open-ended questionse. Gain familiarity with respondentsf. Test fieldwork arrangementsg. Train and test fieldworkersh. Estimate response ratei. Estimate interview etc. timej. Test analysis procedures


chapter 10 questionnaire surveys: typology, design and coding

Documents

sport studies

sport management

roles of questionnaire

household questionnaire

definitions questionnaire

routledge slide

research methods

practical guide