THE ZEN OF SURVEYS

Heather Morrison

Communication 801, February 2010

This work is licensed under the Creative Commons Attribution-Noncommercial-Share Alike 2.5 Canada License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-sa/2.5/ca/ or send a letter to Creative Commons, 171 Second Street, Suite 300, San Francisco, California, 94105,.USA

HTTP://WWW.SURVEYMONKEY.COM

CREATE NEW SURVEY

EDIT SURVEY

ADD A QUESTION - EXAMPLES

MULTIPLE CHOICE MULTIPLE ANSWERS

PREVIEW SURVEY

COLLECT RESPONSES

NEXT STEPS

HTTP://WWW.SURVEYMONKEY.COM/S/GD37BTY

ANALYZE RESULTS

WHY SURVEYS?

Data about groups of people Opinions Factual information Attitudes, values

Classroom tool (online polls) Secondary analysis Known method / advocacy tool

FRED HORNE, ALBERTA MLA ON CONSULTATION SESSIONS RE RECENT 39% INCREASE IN FUNDING FOR PUBLIC LIBRARIES:

“Through our consultation we encountered so many Albertans who are passionate and committed to providing quality library services. They recognized that libraries are a fundamental and tremendously important part of community life. They acknowledged that their lives as citizens and collectively as a community are enhanced because of the programs and services they access at their library. Their dedication is both remarkable and infectious. We hope we captured their passion in our report”.

PLANNING A SURVEY

The research questionAre BC ELN communications as effective as they could be?

POPULATION AND SAMPLE

Census – full population Sampling frame: everyone who could

respond Probability sampling

Random numbers table Systematic Multi-stage cluster

Non-probability sampling

UNIT OF ANALYSIS

Individual, family, journal, newspaper You

Singular Plural: family, social group, community

BC ELN Office / network Individual / department / library Functional groups

FROM CONCEPT TO INDICATOR (DE VAUS)

Clarify concepts BC ELN communications: website, listservs,

quarterly newsletter, budget document, annual Year in Review

Effective: key stakeholders have the information they need (but not unwanted information)

Develop indicators Importance, satisfaction, perceived gaps

Evaluate initial indicators

DECIDE ON APPROACH

Questionnaire Web-based Mail Telephone

Focus group Interview (structured or semi-structured)

Telephone In-person Consumer society / agonistic (Brinkmann, Kvale)

Content analysis

DESIGNING A QUESTIONNAIRE

Length Introduction: purpose of questionnaire Question order

Easy to hard Logical

Thank you & follow-up Pretesting

TO ANSWER A QUESTION…

Decoding the question How often do you watch television? Memory Analysis Opinion. Do I have one? Social desirability bias

CREATING QUESTIONS

Clear, unambiguous wording Neutral non-leading questions (??) Easy response options

Preselected How often do you watch television

1-2 hours / month 1-2 hours / day

Right amount Includes the responses people want to give

(don’t know) Sensitive questions

Leading?

TYPES OF QUESTIONS

Agree / disagree Scales / Likert

Strongly agree to strongly disagree Ranking Open versus closed Filtering questions

ANALYSIS & INTERPRETATION

Response rate Differences between responders and non-

responders A tale of two surveys

Reliability Opinion / multi-question checking

Generalizability (sample) Validity Coding (open-ended questions)

CRITICAL EVALUATION

Bias: funders?

Introduction and question wording

Given the current tough economic climate, how much funding should be given to the arts?

Versus: Given how much the arts enhance our lives,

how much funding should this area receive?

CRITICAL EVALUATION: RESPONSES

Self-selection bias Open web surveys Waltham study

Who was included? Who was excluded?

Marginalized groups Homeless Illegal immigrants

Response rates

BC ELN DRAFT COMMUNICATIONS SURVEY

DISCUSSION QUESTIONS

Consumer versus agonistic interviews Experiences with surveys Video: Ask a Silly Question

REFERENCES

Brinkmann, S. (2007). Could interviews be epistemic? an alternative to qualitative opinion polling. Qualitative Inquiry, 13(8), 1116-1138. doi:10.1177/1077800407308222

De Vaus, D. A. (1995). Surveys in social research (4th ed. ed.). North Sydney, NSW, Australia : Allen & Unwin.

Kvale, S. (2006). Dominance through interviews and dialogues. Qualitative Inquiry, 12(3), 480-500. doi:10.1177/1077800406286235

REFERENCES

Waltham, M. 2009. The future of scholarly journal publishing among social sciences and humanities journals. Princeton, New Jersey. Retrieved from: http://www.nhalliance.org/bm~doc/hssreport.pdf

- critique of Waltham study: Morrison, H. 2009. Humanities and social sciences: thoughts towards transition to OA. The Imaginary Journal of Poetic Economics. Retrieved from: http://poeticeconomics.blogspot.com/2009/09/humanities-and-social-sciences-thoughts.html