1. Types of research studies 1 CASE STUDIES EXPERIMENTS DESIGN STUDIES SURVEYS Survey Research (van Aalst)

2. Goals of survey research Measure or estimate a population on one or more variables How many households have a computer < 3 years old, for different strata of income and education levels? How many classrooms have more than one computer, for schools in areas with different socio-economic status? What is the distribution of income from work, for Hong Kong, Taipei, and Singapore? Survey Research (van Aalst) 2 4.Surveys 3. Example 2006 New Zealand Census Survey Research (van Aalst) 3 4.Surveys 4. TIMSS 2011 Survey Research (van Aalst) 4 4.Surveys 5. Survey methods Generally uses descriptive statistics for testing hypotheses or answering research questions Can be done via questionnaires, interviewing, and observation Questionnaires most common Interview and observation data need to be quantified What are the advantages of each method? Survey Research (van Aalst) 5 4.Surveys 6. Sampling Sometimes whole population is measured All taxpayers in Hong Kong All persons entering Hong Kong by air Most often a representative and randomly selected sample is used and one makes inferences about the population from that sample Survey Research (van Aalst) 6 4.Surveys 7. Representative sample Are the following representative samples of their populations? The men and women in this course as a sample of all men and women taking M. Sc. courses in Hong Kong? People in Hong Kong who have a broadband connection to the Internet as a sample of all people in Hong Kong? Survey Research (van Aalst) 7 4.Surveys 8. Random sample Each member of the population has an equal chance being selected for the survey Maximizes the chance that patterns in the sample match those in the population Statistically useful Mean value and variation in the population can be estimated from knowing the mean value and variation of the random sample When the sample is not random this kind of estimation is less accurate Survey Research (van Aalst) 8 4.Surveys 9. Problems with representativeness and randomness Low response rates, say 20% sub-populations may be systematically excluded Systematic bias A random sample is no longer random or representative when some students do not turn up for a test Sampling % of total population may over- represent major urban centers and under- represent rural areas Survey Research (van Aalst) 9 4.Surveys 10. Planning a questionnaire Can use a flow chart technique to plan the sequencing of questions Be able to anticipate the type & range of responses that their questions are likely to elicit Survey Research (van Aalst) 10 4.Surveys 11. A flow chart technique for question planning Survey Research (van Aalst) 11 4.Surveys 12. Operationalizing the questionnaire Clear purpose Clear on what needs to be included or covered Exhaustive coverage of elements Asks most appropriate kinds of question Elicits most appropriate kinds of data to answer the research purposes Asks for empirical data Survey Research (van Aalst) 12 4.Surveys 13. Avoiding pitfalls in question writing Avoid leading questions which suggests that there is only one acceptable answer Do you prefer abstract, academic-type courses, or practical courses that have some pay-off in your day-to-day teaching? Survey Research (van Aalst) 13 4.Surveys 14. Avoiding pitfalls in question writing Avoid complex questions Would you prefer a short, non-award bearing course (3, 4 or 5 sessions) with part-day release (e.g. Wednesday afternoons) and one evening per week attendance with financial reimbursement for travel, or a longer, non-award bearing course (6, 7 or 8 sessions) with full-day release, or the whole course designed on part-day release without evening attendance? Survey Research (van Aalst) 14 4.Surveys 15. Avoiding pitfalls in question writing Avoid irritating questions or instructions Have you ever attended an in-service course of any kind during your entire career? Avoid questions that use negatives and double negatives How strongly do you feel that no teacher should enroll on the in-service, award- bearing course who has not completed at least two years full-time teaching? Survey Research (van Aalst) 15 4.Surveys 16. Avoiding pitfalls in question writing Avoid ambiguity in words Does your child regularly do homework? (Once a day, once a year, once a term, once a week?) How many computers do you have in school? (Present but broken; out of school being repaired, staffs or students?) Survey Research (van Aalst) 16 4.Surveys 17. Avoiding pitfalls in question writing Avoid any overlap of categories. How old are you? 15-20 20-30 30-40 40-50 Survey Research (van Aalst) 17 4.Surveys 18. Cover letters/sheets and follow-up letters Provide title for project State purpose and expected benefits Mention sponsorship or permission for the research Mention incentives for completion Provide assurances of confidentiality, anonymity and non-traceability Thank respondents in advance for their co-operation Survey Research (van Aalst) 18 4.Surveys 19. Piloting the questionnaire Check clarity of questionnaire items, instructions, and layout Gain feedback on validity of items Eliminate ambiguities Check time to complete Identify misunderstood or non- completed items Check if it is too long or too short, too easy or too difficult Survey Research (van Aalst) 19 4.Surveys 20. Further reading Fowler, Jr., F. J. (2009). Survey research methods, 4th edition. Thousand Oaks, CA: SAGE. Survey Research (van Aalst) 20