designing surveys to determine the impact of online social networks #naspatech8
DESCRIPTION
As the popularity of online social networking sites continues to grow with incoming students, higher education needs to have a better understanding of online social media tools. An understanding of how to design surveys is needed to determine the impact of online social networks on student’s social and academic interactions. Student affairs professionals spend considerable time analyzing, reporting, and discussing survey results. We often hear during these conversations, how do we ensure this information is accurate? Is this information generalizable? Is this information meaningful? Did we ask about student usage? Should we have asked about student perceptions of online social networks? How does online social networking impact students’ academic and social interactions on campus? Should we have asked about privacy? This presentation will address these questions and others by outlining the steps in survey development, posing areas to explore in online social networks, and engaging attendees in the survey construction process.TRANSCRIPT
![Page 1: Designing Surveys to Determine the Impact of Online Social Networks #naspatech8](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558cde31d8b42a1b5a8b45d4/html5/thumbnails/1.jpg)
Designing Surveys to Determine Impact of Online
Social Networks on Student’s Social and Academic
Interactions
Nicola Ritter, M.Ed.
Data Analyst, Student Life Studies
@nicolalritter #naspatech8
![Page 2: Designing Surveys to Determine the Impact of Online Social Networks #naspatech8](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558cde31d8b42a1b5a8b45d4/html5/thumbnails/2.jpg)
Take Away Points
I. Outline the facets of survey development
II. Pose areas to explore in online social networks (OSN)
III. Engage in the survey construction process
![Page 3: Designing Surveys to Determine the Impact of Online Social Networks #naspatech8](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558cde31d8b42a1b5a8b45d4/html5/thumbnails/3.jpg)
I. Tailored DesignDillman, Smyth, & Christian (2009)
1. Reduces Survey Error
2. Develops procedures that are respondent-centered
3. Includes variety of features that build positive social exchange
![Page 4: Designing Surveys to Determine the Impact of Online Social Networks #naspatech8](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558cde31d8b42a1b5a8b45d4/html5/thumbnails/4.jpg)
1. Reducing Survey Error Dillman, Smyth, & Christian (2009)
1. Coverage Error – when some members of the population are excluded AND those excluded are different from those included (e.g. students with text messaging vs. students without text messaging)
2. Sampling Error – surveying only a portion of the population (e.g. full time vs. part-time students)
(ie. Ravert, Calix, & Sullivan (2010)
![Page 5: Designing Surveys to Determine the Impact of Online Social Networks #naspatech8](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558cde31d8b42a1b5a8b45d4/html5/thumbnails/5.jpg)
1. Reducing Survey Error Dillman, Smyth, & Christian (2009)
3. Non-response Error – respondents different from non-respondents (e.g. missing data)
4. Measurement Error – inaccurate or imprecise responses (e.g. item construction)
(ie. Ravert, Calix, & Sullivan (2010))
![Page 6: Designing Surveys to Determine the Impact of Online Social Networks #naspatech8](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558cde31d8b42a1b5a8b45d4/html5/thumbnails/6.jpg)
2. Develops procedures that are respondent-centered Dillman, Smyth, & Christian (2009)
1. Increase the Benefits of Participation
2. Decrease Cost of Participation
3. Establish Trust
![Page 7: Designing Surveys to Determine the Impact of Online Social Networks #naspatech8](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558cde31d8b42a1b5a8b45d4/html5/thumbnails/7.jpg)
Increase the Benefits of Participation
• Provide information about the survey
• Ask for help or advise
• Show positive regard
• Say thank you
• Support group values
• Give tangible rewards
• Make the questionnaire interesting
• Provide social validation
• Inform people that opportunities to respond are limited
![Page 8: Designing Surveys to Determine the Impact of Online Social Networks #naspatech8](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558cde31d8b42a1b5a8b45d4/html5/thumbnails/8.jpg)
Decrease Cost of Participation
• Make it convenient to respond• Avoid subordinating language• Make the questionnaire short and easy to
complete• Minimize requests to obtain personal or
sensitive information• Emphasize similarity to other requests or task
to which a person has already responded
![Page 9: Designing Surveys to Determine the Impact of Online Social Networks #naspatech8](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558cde31d8b42a1b5a8b45d4/html5/thumbnails/9.jpg)
Establish Trust
• Obtain sponsorship by legitimate authority
• Provide a token of appreciation in advance
• Make the task appear important
• Ensure confidentiality and security of information
![Page 10: Designing Surveys to Determine the Impact of Online Social Networks #naspatech8](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558cde31d8b42a1b5a8b45d4/html5/thumbnails/10.jpg)
3. Includes variety of features that build positive social exchangeDillman, Smyth, & Christian (2009)
Outside the Survey• Invitations• Reminder s• Thank yous• Follow-up that offers results
Inside the Survey• Clickable graphics• Analog Scales
![Page 11: Designing Surveys to Determine the Impact of Online Social Networks #naspatech8](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558cde31d8b42a1b5a8b45d4/html5/thumbnails/11.jpg)
II. Research Questions: Beyond Usage(Ritter, in review)
1. Insight to student perspectives (Martinez Aleman & Wartman, 2009)
2. Social and Academic Interactions– Student/Student, Student/Faculty-Staff Interactions (Kord & Wolf-
Wendel, 2009; Teclehaimanot & Hickman, 2011)
– Student/Organization Interactions– Impact of New Features (ie. Groups/Circles, geo-location tools)
3. Privacy (Eberhardt, 2005; Eberhardt, 2006; Koleck & Saunders, 2008; Kirschner & Karpinski, 2010)
4. Student Affair Professionals Usage– Students– Organization– Other
![Page 12: Designing Surveys to Determine the Impact of Online Social Networks #naspatech8](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558cde31d8b42a1b5a8b45d4/html5/thumbnails/12.jpg)
Brainstorm Topic/subtopics
![Page 13: Designing Surveys to Determine the Impact of Online Social Networks #naspatech8](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558cde31d8b42a1b5a8b45d4/html5/thumbnails/13.jpg)
Overall Survey Construction
1. What survey mode(s) will be used to ask the questions?
2. Is this question being repeated from another survey, and/or will answers be compared to previously collected data?
3. Will respondents be willing and motivated to answer accurately?
4. What type of information is the question asking for?
![Page 14: Designing Surveys to Determine the Impact of Online Social Networks #naspatech8](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558cde31d8b42a1b5a8b45d4/html5/thumbnails/14.jpg)
Guidelines for Word Choice & Question Format (Dillman, et. al. (2009)
1. Make sure the question applies to the respondent.
2. Make sure the question is technically accurate.
3. Ask one question at a time.
4. Use similar and familiar words
5. Use specific and concrete words to specify the concepts clearly.
![Page 15: Designing Surveys to Determine the Impact of Online Social Networks #naspatech8](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558cde31d8b42a1b5a8b45d4/html5/thumbnails/15.jpg)
Guidelines for Word Choice & Question Format (Dillman, et. al. (2009)
6. Use as few words as possible to pose the question.
7. Use complete sentences with simple sentence structures.
8. Make sure “yes” means yes and “no” means no.
9. Be sure the question specifies the response task.
![Page 16: Designing Surveys to Determine the Impact of Online Social Networks #naspatech8](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558cde31d8b42a1b5a8b45d4/html5/thumbnails/16.jpg)
1. Make sure the question applies to the respondent.
A question that does not require an answer from every respondent
If you have a Facebook profile, about how many hours per day do you spend on Facebook?
hours per day
![Page 17: Designing Surveys to Determine the Impact of Online Social Networks #naspatech8](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558cde31d8b42a1b5a8b45d4/html5/thumbnails/17.jpg)
1. Make sure the question applies to the respondent.
A question that uses a filter
Do you have a Facebook profile?
Yes
No
About how many hours per day do you spend on Facebook?
hours per day
![Page 18: Designing Surveys to Determine the Impact of Online Social Networks #naspatech8](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558cde31d8b42a1b5a8b45d4/html5/thumbnails/18.jpg)
2. Make sure the question is technically accurate.
Which Facebook features/activities listed below do you use? (Check all that apply)
Mini-feed
News feed
Check-in
Posting on other people’s walls
Searching for people
![Page 19: Designing Surveys to Determine the Impact of Online Social Networks #naspatech8](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558cde31d8b42a1b5a8b45d4/html5/thumbnails/19.jpg)
2. Make sure the question is technically accurate.
Which Facebook features listed below do you use? (Check all that apply)
News feed
Check-in on Places
Wall-to-Wall
Search bar
![Page 20: Designing Surveys to Determine the Impact of Online Social Networks #naspatech8](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558cde31d8b42a1b5a8b45d4/html5/thumbnails/20.jpg)
3. Ask one question at a time.
Two questions in one
From which one of these sources did you first learn about A&M moving to the SEC?
University website
Someone at work
While traveling to campus
![Page 21: Designing Surveys to Determine the Impact of Online Social Networks #naspatech8](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558cde31d8b42a1b5a8b45d4/html5/thumbnails/21.jpg)
3. Ask one question at a time.
From which one of these sources did you first learn about A&M moving to the SEC?
University website
Radio
TexAgs
Another person
Where were you when you first heard about it?
At work
At home
Traveling to work
Somewhere else:
![Page 22: Designing Surveys to Determine the Impact of Online Social Networks #naspatech8](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558cde31d8b42a1b5a8b45d4/html5/thumbnails/22.jpg)
4. Use similar and familiar words
From which one of these sources did you first learn about A&M moving to the SEC?
University website
Radio
TexAgs
Another person
![Page 23: Designing Surveys to Determine the Impact of Online Social Networks #naspatech8](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558cde31d8b42a1b5a8b45d4/html5/thumbnails/23.jpg)
4. Use similar and familiar words
From which one of these sources did you first learn about Texas A&M moving to the Southeastern Conference?
University website
Radio
TexAgs
Another person
![Page 24: Designing Surveys to Determine the Impact of Online Social Networks #naspatech8](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558cde31d8b42a1b5a8b45d4/html5/thumbnails/24.jpg)
5. Use specific and concrete words to specify the concepts clearly.
How many times did you log on to Facebook as a student affairs professional last week?
# of times
![Page 25: Designing Surveys to Determine the Impact of Online Social Networks #naspatech8](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558cde31d8b42a1b5a8b45d4/html5/thumbnails/25.jpg)
5. Use specific and concrete words to specify the concepts clearly.
How many times did you log on to Facebook as a student affairs professional last week?
# of times
To work with students
![Page 26: Designing Surveys to Determine the Impact of Online Social Networks #naspatech8](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558cde31d8b42a1b5a8b45d4/html5/thumbnails/26.jpg)
5. Use specific and concrete words to specify the concepts clearly.
How many times did you log on to Facebook as a student affairs professional to connect with students last week?
# of times
![Page 27: Designing Surveys to Determine the Impact of Online Social Networks #naspatech8](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558cde31d8b42a1b5a8b45d4/html5/thumbnails/27.jpg)
6. Use as few words as possible to pose the question.
According to Strauss and Howe (1992), the United States is made up of different generations. Based on their definitions, which generation do you belong to?
Silent Generation (born in years 1924 through 1942)
Boom Generation (born in years 1943 through 1960)
Generation X (born in years 1961 through 1981)
Millennial Generation (born in years 1982 through 2002)
![Page 28: Designing Surveys to Determine the Impact of Online Social Networks #naspatech8](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558cde31d8b42a1b5a8b45d4/html5/thumbnails/28.jpg)
6. Use as few words as possible to pose the question.
Which age range were you born?
1924 - 1942
1943 -1960
1961 - 1981
1982 - 2002
What year were you born?
![Page 29: Designing Surveys to Determine the Impact of Online Social Networks #naspatech8](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558cde31d8b42a1b5a8b45d4/html5/thumbnails/29.jpg)
7. Use complete sentences with simple sentence structures.
Number of years lived in Texas
Years
Your city or town
City or Town
![Page 30: Designing Surveys to Determine the Impact of Online Social Networks #naspatech8](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558cde31d8b42a1b5a8b45d4/html5/thumbnails/30.jpg)
7. Use complete sentences with simple sentence structures.
Number of years lived in Texas
Years
Your city or town
City or Town
About 25% of respondents listed the # of years they lived in the city or town
![Page 31: Designing Surveys to Determine the Impact of Online Social Networks #naspatech8](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558cde31d8b42a1b5a8b45d4/html5/thumbnails/31.jpg)
7. Use complete sentences with simple sentence structures.
How many years have you lived in Texas?
Years
In what city or town do you live?
Name of City or Town
![Page 32: Designing Surveys to Determine the Impact of Online Social Networks #naspatech8](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558cde31d8b42a1b5a8b45d4/html5/thumbnails/32.jpg)
8. Make sure “yes” means yes and “no” means no.
A question with a double negative
Do you favor or oppose not allowing Texas A&M to use information from Facebook and Twitter during proceedings?
Favor
Oppose
![Page 33: Designing Surveys to Determine the Impact of Online Social Networks #naspatech8](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558cde31d8b42a1b5a8b45d4/html5/thumbnails/33.jpg)
8. Make sure “yes” means yes and “no” means no.
Revision with no double negative
Do you favor or oppose allowing Texas A&M to use information from Facebook and Twitter during proceedings?
Favor
Oppose
![Page 34: Designing Surveys to Determine the Impact of Online Social Networks #naspatech8](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558cde31d8b42a1b5a8b45d4/html5/thumbnails/34.jpg)
8. Make sure “yes” means yes and “no” means no.
Revision with that preserves important wording
In the upcoming student elections, you will be asked to vote on this referendum: “[Insert referendum]” If the election were held today, would you vote for or against approval?
For
Against
![Page 35: Designing Surveys to Determine the Impact of Online Social Networks #naspatech8](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558cde31d8b42a1b5a8b45d4/html5/thumbnails/35.jpg)
9. Be sure the question specifies the response task.
A question specifying different tasks
How many days a week do you log in to Facebook?
More than once a day
Once a day
A few times a week
Once every other week
Once a month
Less than once a month
![Page 36: Designing Surveys to Determine the Impact of Online Social Networks #naspatech8](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558cde31d8b42a1b5a8b45d4/html5/thumbnails/36.jpg)
9. Be sure the question specifies the response task.
Revision
How many days per week do you log in to Facebook?
days per week
![Page 37: Designing Surveys to Determine the Impact of Online Social Networks #naspatech8](https://reader035.vdocuments.mx/reader035/viewer/2022062708/558cde31d8b42a1b5a8b45d4/html5/thumbnails/37.jpg)
ReferencesDillman, D. A., Smyth, J.D., & Christian, L. M., (2009). Internet, Mail, and Mixed-Mode Surveys: The Tailored Design Method (3rd ed.). Hoboken, NJ: John Wiley and Sons, Inc.
Eberhardt, D. (2005). Should institutions respect students’ on-line privacy of facebook?. Journal of College and Character, 6(7).
Eberhardt, D. (2006). The facebook/myspace era: A help or hinderance to college students and administrators?. Journal of College and Character, 7(7).
Kirschner, P. A., and Karpinski, A. C. (2010). Facebook and academic performance, Computers in Human Behavior, 26(6), 1237-1245. doi: 10.1016/j.chb.2010.03.024
Kolek, E. A. & Saunders, D. (2008). Online disclosure: An empirical examination of undergraduate facebook profiles. NASPA Journal, 45 (1), 1-25.
Kord, J., & Wolf-Wendel, L. (2009). The relationship between online social networking and academic and social integration. College Student Affairs Journal, 28(1), 103.
Martinez-Aleman, A. M. and Wartman, K. L. Online Social Networking on Campus: Understanding What Matters in Student Culture, Routledge, New York, NY, 2009.
Ravert, R. D., Calix, S. I., Sullivan, M. J. (2010). Research in brief: Using mobile phones to collect daily experience data from college undergraduates. Journal of College Student Development, 51 (3) 343-352.
Ritter, N. (in review). Journal sponsored by Student Affairs Administrators in Higher Education.
Teclehaimanot, B., & Hickman, T. (2011). Student-teacher interaction on facebook: What students find appropriate. TechTrends: Linking Research and Practice to Improve Learning, 55(3), 19-30.