challenges of surveys in kenya presentation by jack abebe

1Saturday, April 8, 2023

INTRODUCTION TO SURVEYS Definition- What is a survey

Purpose of surveys

Types of surveys

Planning surveys

Characteristics of surveys

Challenges in conducting surveys in Kenya

“Research is a craft – It takes practice and more practice”

Saturday, April 8, 2023 2

A survey is a series of questions asked of a group of people in order to gain information

Information gathered can be facts, attitudes, feelings, beliefs

Saturday, April 8, 2023 3Saturday, April 8, 2023 3

Can complete structured questions with many stakeholders within a relatively short time frame.

Can be completed by telephone, mail, fax, or in-person.

It is quantifiable and generalizable to an entire population if the population is sampled appropriately.

Saturday, April 8, 2023 4

Standardized, structured questionnaire minimizes interviewer bias.

Tremendous volume of information can be collected in short period of time.

Can take less time to analyse than qualitative data.

Saturday, April 8, 2023 5

Satisfaction survey Evaluations Applications Voting Invitations

Saturday, April 8, 2023 6

The goal is NOT to “do a survey” The goal is to acquire data to make proper

decisions A survey is only one channel of information

in support of that decision-making process

Saturday, April 8, 2023 7

The goal of any survey is to collect data which can be analyzed, and used to aid decisions

Saturday, April 8, 2023 8

Deciding on a research question Choosing the format of your questions Choosing the format of your interview--if you use

an interview Editing your questions Sequencing your questions Refining your survey instrument Choosing a sampling strategy

Saturday, April 8, 2023 9

Map out questions to help you gain the information needed to meet the survey objective

Question Types:

Open: No pre-set selections – type in text, numbers – whatever is appropriate for the

question. Open-ended questions – unstructured fill-in-the-blank types of questions. Good for gathering general information and revealing what is on the respondents’

minds Open-ended questions require a lot of time to analyze and are more readily open

to misinterpretation.

Closed: Choose from pre-set selections only Multiple choice, Yes/No, True/False (etc.) Choices are structured; respondents must choose from a scale or a series of

options. Good for nailing down specific, numeric options, and they’re easier to analyze They may not offer enough choices and options for people to truly express their

opinionsMixed: Combinations of closed and open (i.e. “Other – Please specify_________”)

Saturday, April 8, 2023 10

Plan the survey Design the survey Test the survey/ Piloting Deploy the survey Reporting

Saturday, April 8, 2023 11

Saturday, April 8, 2023 12

Objectives:

What information do you need to define?How many will attend civic education and how many of this group will eat lunch.

What decisions will this information impact?How much space is needed for the participants and how many lunches are needed.

Who is the audience for the survey?Every Woman Aspirant in Kisii County

Who is the audience for the report?CREAW

In what format is the report needed?Excel spreadsheet with names and numbers or head count

How will you deliver the survey to your audience?

Email

I will attend the civic education programme?

Decisions regarding the protocol for the surveys are based on the type of information being sought, the budget available for the project, timing considerations, and the target population.

The method of administration will affect costs and response rate, and will also influence which questions may be asked and how they are asked.

Saturday, April 8, 2023 13

Informed consent Clear instructions Debriefing Confidentiality

Saturday, April 8, 2023 14

Survey deployment is one of the last things done in any survey project.

Surveys cannot be deployed until the survey design has been tested, databases and tables have been tested, and the data has been checked to ensure that it is being properly collected.

Once surveys have been designed and rigorously tested, they will be ready for deployment.

Saturday, April 8, 2023 15

E-mail Invitation

◦ The invitation is computer generated ◦ E-mail addresses can be imported from GroupWise or a .txt file◦ When a user finishes the survey, this user gets flagged in the database.◦ When necessary, e-mail a second invitation. Users who are flagged will not get a

reminder. ◦ Passwords, links, and instructions can be in the e-mail

E-Mail Survey◦ The survey will be in the body of the e-mail

Print a paper copy of the survey

Handheld device, PDA face to face interviews, telephone interviews Post a link to your survey on the your county/department website.

Saturday, April 8, 2023 16

Do you know what your population is? How can you get a representative sample of

that population? Will non response bias prevent you from

generalizing your results to that population?

Saturday, April 8, 2023 17

Export data to other software applications◦ Access◦ Excel◦ PowerPoint◦ SPSS

Loading responses into a personal database◦ From e-mail◦ From a file (TSV, Tab Separated Value)◦ Using entry screen for interviews or paper-based results◦ From handheld device

Saturday, April 8, 2023 18

Frequency Analysis Time Series Analysis Gap Analysis Banner Report Frequency Mean Pair 360 Degree Report Open Ended Analysis

Saturday, April 8, 2023 19

Saturday, April 8, 2023 20

Answers May Be Inaccurate Because: People don’t know the causes of their own

behavior People’s memories are inaccurate People are not good at predicting their future

behavior People are not telling you what they know

because of◦ Social desirability bias◦ Demand characteristics◦ Response sets

Saturday, April 8, 2023 21

Information collected;◦ May not have internal validity◦ May not have construct validity because of self-

report problems◦ May not have external validity because of poor

sampling or because of non response bias◦ May not answer research question because

survey questions weren’t focused on hypotheses

Saturday, April 8, 2023 22

Financial Constraints-surveys are usually expensive and requires heavy finances to execute.

This is due to larger coverage in terms of the populations under survey.

Saturday, April 8, 2023 23

Suspicion and resistance- There is usually a height of suspicion which translates into resistance and reluctance in giving information by respondents.

This can be managed by clarity of purpose.

Saturday, April 8, 2023 24

Requires High Human Resource Input to execute

In-depth training in the system and ongoing development for all personnel may seem excessive;

A knowledgeable group of people is much more efficient to work with

Saturday, April 8, 2023 25

Capacity issues are often glaringly obvious But they can also be somewhat discreet and

difficult to detect. It is much more difficult to mitigate capacity

issues mid-way through a project, especially one that is working to a tight

budget, than at the beginning

Saturday, April 8, 2023 26

More difficult to collect a comprehensive understanding of respondents’ perspective (in-depth information) compared to in-depth interviews

or focus groups.

Saturday, April 8, 2023 27

Survey research is the most used research method

Survey research is the most misused research method

You know how to use rather than abuse survey research

Saturday, April 8, 2023 28

Vollmer, WM., Osborne, ML., et.al. Recruiting hard-to-reach subjects: Is it worth the effort? Controlled Clinical Trials. 15(2):154-9 1994.

Nutbeam, D., Smith, C. et.al. Maintaining evaluation designs in long term community based health promotion programmes: Heartbeat Wales Case Study.

Journal of Epidemiology and Community Health. 47(2):127-33, 1993.

Saturday, April 8, 2023 29

Anker, M. Guidotti, R.J. et.al. Rapid evaluation methods (REM) of health services performance: Methodological observations. Bulletin of the World Health Organization. 71(1):15-21, 1993.

Harlow, B.L., Crea, E.C., et.al. Telephone answering machines: The influence of leaving messages on telephone interviewing response rates. Epidemiology. 4(4):380-3 1993.

Saturday, April 8, 2023 30

THE END

THANK YOU FOR YOUR ATTENTION

Questions are Welcome

Saturday, April 8, 2023 31