presentation on research methodologies

Training on“Research Methodologies”

(20-24 December 2010)

DEFINING RESEARCH

• Research is an ORGANIZED and SYSTEMATIC way of FINDING ANSWERS to QUESTIONS

KEY WORDS

• SYSTEMATIC because there is a definite set of procedures and steps which you will follow. There are certain things in the research process which are always done in order to get the most accurate results.

(Continued)

• ORGANIZED in that there is a structure or method in going about doing research. It is a planned procedure, not a spontaneous one. It is focused and limited to a specific scope.

(Continued)

• FINDING ANSWERS is the end of all research. Whether it is the answer to a hypothesis or even a simple question, research is successful when we find answers. Sometimes the answer is no, but it is still an answer.

Key Words (Continued)

• QUESTIONS are central to research. If there is no question, then the answer is of no use. Research is focused on relevant, useful, and important questions. Without a question, research has no focus, drive, or purpose.

What is the value of research?

Types of Research

Some commonly used categories are:

• Primary Research Secondary Research • Pure Research Applied Research

• Scientific Research Social Research

• Experimental Research Evaluative Research

Primary ResearchPrimary research (also called field research) involves the collection of data that does not already exist, which is research to collect original data. Primary Research is often undertaken after the researcher has gained some insight into the issue by collecting secondary data. This can be through numerous forms, including questionnaires, direct observation and telephone interviews amongst others. This information may be collected in things like questionnaires and interviews .

Secondary ResearchSecondary research (also known as desk research) involves the summary, collation and/or synthesis of existing research rather than primary research, where data is collected from, for example, research subjects or experiments.

Pure/Basic ResearchBasic research or fundamental research (sometimes pure research) is research carried out to increase understanding of fundamental principles.

Many times the end results have no direct or immediate commercial benefits: basic research can be thought of as arising out of curiosity. However, in the long term it is the basis for many commercial products and applied research.

Basic research is mainly carried out by universities.

Applied Research

• Solve specific problems help practitioners. Market new product.

• Choose one policy over the other.

• For improving productivity problem with machines, raw material, persons working.

Types of applied research- Action research: Those who are being

studied participate in research process; research incorporates popular knowledge; focus on power with goal of empowerment increase awareness; tied to political action

- Impact Assessment: Estimate the likely consequences of planned change.

- Evaluation Research: Did the program work? Measures the effectiveness of program.

The Eleven Steps in a Research Project

• Define the topic

• Find out what is known about the topic

• Clarify concepts and their measurement

• Establish an appropriate data collection method

• Operationalize concepts and design the research/data collection instruments

• Select a sample of subjects/respondents

• Consider the purpose, value and ethics of the study

• Collect the data

• Process the data

• Analyse the data

• Present the results

Research Techniques

• Experimental technique.

• Surveys. Quantitative

• Content analysis.

• Use of existing statistics.

• Field research.• Case study. Qualitative

• Focus group discussions» Mixed methods/techniques

VARIABLE

Variable• A central idea in research.• Variable is a concept that varies.• Anything (concept/term) that can take

on differing or varying values. Could be numerical.

• Variation can be in quantity, intensity, amount, or type.

• Examples: Production units, Absenteeism, Gender, Religion, Motivation, Grade, Age.

HYPOTHESIS

BackgroundOnce variables identified establish the relationship through logical reasoning.

Proposition.•Proposition is a statement about variables judged to be true.

Definition• Hypothesis is a testable

counterpart of proposition i.e. If variables refer to empirical reality then it could be testable. Therefore

• Hypothesis is a testable proposition.

Example 1

• Level of job commitment of the Officers is associated with their level of efficiency.

Example –2

• Level of job commitment of the Officers is positively associated with their level of efficiency.

Example – 3 • The higher the level of job

commitment of Officers the lower their level of absenteeism.

TOOLS FOR DATA COLLECTION

Data Collection Methods• Quantitative Methods: Findings can be reduced to

numerical summaries; relatively more objective and ‘scientific’

• Qualitative Methods: Findings are expressed in words, narrative and not numbers; more naturalistic and open, relies on interpretation and emphasizes ‘meanings’.

• Participatory Methods: Close to qualitative research but less formal; exclusive to development work; attempts to correct the imbalance between researcher and subjects of the research.

Quantitative, Qualitative & Participatory Methods in Use

Quantitative Quantitative MethodsMethods

Qualitative Qualitative MethodsMethods

Participatory Participatory MethodsMethods

•SurveySurvey•ExperimentsExperiments

•Focus GroupFocus Group•Interviews Interviews (Semi-(Semi-structured/in-structured/in-depth)depth)•Field Field ObservationsObservations•Case StudiesCase Studies•EthnographyEthnography

•MappingMapping•Ranking and Ranking and ScoringScoring•Transect WalkTransect Walk•Seasonal Seasonal CalendarsCalendars

SURVEY

Survey

– Survey is a method of collecting data in which a specifically defined group of individuals are asked to answer a number of identical questions to produce quantitative datasets, amenable to computer-based analysis.

Types of Survey• Demographic Surveys

• Health & Nutrition Surveys

• Opinion Surveys

• KAP Surveys

• Rapid Assessment Surveys

Successful Surveys• Depend on

– Proper Sampling– Questions Asked/Questionnaires– Administering Questionnaires/Interviewing – Data Processing and Frame of Analysis

SAMPLE AND SAMPLING TERMINOLOGY

Sampling

Sampling is the process of selecting units (e.g., people, organizations) from a population of interest so that by studying the sample we may fairly generalize our results back to the population from which they were chosen.

Key Sampling Terms

• The group you wish to generalize your findings to is called the population in your research.

• The listing of the accessible population from which you'll draw your sample is called the sampling frame.

• The sample is the group of people who you select to be in your study based on some principles of sampling.

Sampling Methods

• A probability sampling method is any method of sampling that utilizes some form of random selection. With a probability sample, we know the odds or probability that we have represented the population well. – Simple Random Sampling, Systematic Random

Sampling, Cluster Sampling, Stratified Sampling

• Non-probability sampling does not involve random selection; with non-probability samples, we may or may not represent the population well. – Convenience Sampling, Snowball Sampling, Quota

Sampling, Expert Sampling

Sample Size

• Depends on

– Population Size (Little difference beyond 25,000)– Confidence Interval– Confidence Level– Level of variation with respect to the main variable

you’re studying– Type of Survey (For opinion surveys 1000 is

considered appropriate)

Confidence Interval

• If you use a confidence interval of 4 and 47% percent of your sample picks an answer you can be "sure" that if you had asked the question of the entire relevant population between 43% (47-4) and 51% (47+4) would have picked that answer.

Confidence Level

• Tells you how sure you can be about your confidence interval. It is expressed as a percentage and represents how often the true percentage of the population who would pick an answer lies within the confidence interval.

• The 95% confidence level means you can be 95% certain; the 99% confidence level means you can be 99% certain.

• Most researchers use the 95% confidence level.

Sample Size Calculator

• http://www.surveysystem.com/sscalc.htm

QUESTIONNAIRE

Individual Exercise

• You have been asked to develop a questionnaire to measure the accessibility of women and children to safe drinking water in UC XYZ. Write down just 5 substantive questions you’d include in your questionnaire.

Types of Questions

• Open-ended questions state a question and leave room for the respondent to write out or narrate an answer.

• Close-ended questions force the respondent to select a single response from a list.

Close-ended and Open-ended Forms of the Same Question

• D. 18 How would you rate the level of sectarian harmony and tolerance in your union council?

____________________________________________

• D. 18 How would you rate the level of sectarian harmony and tolerance in your union council?

1) Very Good 2) Good 3) Moderate

4) Poor 5) Very poor 6) Do not know

Avoid Double-Barreled Questions

• D-B question consists of two or more questions joined together.

• Makes the answer ambiguous. “Does this Co have pension and health insurance benefits?”

Examples of Double-barreled and Leading Questions

Are you satisfied with the performance of the district and provincial governments?

1. Yes 2. No 3. Not Sure

The chief justice of Pakistan was unconstitutionally deposed on November 2, 2008. Do you think he should be reinstated?

1. Yes 2. No 3. Do not know

Avoid Leading Questions

• LQ: That leads the respondent to choose one response over the other by its wording.

• Make respondents feel that all responses are legitimate.

• “You don’t smoke, do you?” “Don’t you think that women should be empowered?”

Avoid Loaded Questions

• LQ: suggest socially desirable answer or are emotionally charged.

• Should the city government repair all broken roads?

• Question loaded with such material that may hit the emotions of people e.g. Asking a husband – Have you ever been beaten up by your wife? Can be embarrassing.

Avoid Burdensome Questions

• People forget.

• Certain question may make serious demand on respondent’s memory. How did you feel about your brother when you were 6 years old?

8. Arrange Questions Sequentially• Make the questionnaire an integrated

whole. No jumping back and forth.

• Make the opening questions simple and interesting. Arouses respondent’s curiosity.

• Funnel Technique: Moving from broader issues to specific one.

9. Use Filter questions if Needed

• Asking question that doesn’t apply to the subject can be irritating or bias the answer.

• Use filter question: That screens out respondents not qualified the second question.

• Asking about bringing up of one’s children. Prior to that ask if a person is married. If yes is he/she a parent.

Advantages of Mail Questionnaire

• 1 Geographic Flexibility: Reach all corners of the country/world.

• 2. Sample Accessibility: Contact those who may inaccessible.

• 3. Save Time:

• 4. Saves Cost:

• Respondent Convenience:

• Anonymity:

• Standardized Questions

Disadvantages of Mail Questionnaire• 1. Low Response Rate• 2. Low Completion Rate• 3. Increases Cost• 4. Interviewer’s Absence• 5. No Control on Question Order• 6. Cannot use Lengthy Questionnaire• 7. No Control Over Environment• 8. Cannot catch the non-verbal

behavior• 9. Non-Literates cannot Participate

Increasing Response Rate• Cover Letter

• Money Helps

• Interesting Questions

• Follow-Ups

• Preliminary Notification

• Survey Sponsorship

• Other Techniques: Questionnaire format, facilitating the return, postage, personalization.

PILOT TESTING THE QUESTIONNAIRE

Pilot Testing

• Also called pre-testing: Small scale trial run of a particular component.

• Here focus is on trial run of questionnaire.

Involves• Small group selected on convenience.

• Could be 25+

• Similar to the one to be sampled.

• Administering questionnaire exactly as planned often is not possible (Problems in mail questionnaire).

• Usually go for personal interviewing

What aspects to be evaluated?

• Reactions of Respondents

• Discovering errors in the instrument

• Checking the sampling procedure

• Staffing and activities of research team

• Evaluating the procedure for data processing and analysis

FIELDWORK

Interviewers• In-house interviewers: hired by the

researcher

• Field Interviewing Service: A research supplier that specializes in gathering data. Trained interviewers, and supervisors. Edit questionnaires in the field. Certify how the data were collected.

• Interviewers need to be healthy, outgoing, and of pleasing appearance i.e. well groomed and well dressed.

Role of interviewer• The survey interview is a social relationship.• It is a short term secondary social

interaction between two strangers with the explicit purpose.

• Interviewer - Interviewee/respondent.• Structured conversation – interviewer asks

prearranged questions and records answers.

• The role of interviewer is difficult. They obtain cooperation and build rapport, yet remain neutral and objective.

STAGES OF AN INTERVIEW

Making initial contact -- Rapport

• Convince the respondent

• Asslaam-o-Alaykum. My name is ______ and I am working for National Survey Co. We are conducting a survey concerning “women empowerment.” I would like to get a few of your ideas.

• Personal interviewer must carry a letter of authority.

Asking the questions• Ask the questions exactly as they are worded.• Read each question very slowly.• Ask the question in the order in which they are

presented in questionnaire.• Repeat questions that are misunderstood or

misinterpreted.• Information volunteered earlier than the actual question.

Record at proper place. But do ask this question.

Probing• Verbal prompts made by the fieldworker.• Probing needed in 2 types of situations:• 1. When the respondent is to be

motivated to enlarge on, to clarify, or explain the answer.

• 2. When there is rambling.

Probing tactics

• Repetition of the question.

• An expectant pause.

• Repetition of the respondent’s reply.

• Neutral questions or comments.

Recording the responses• Closed ended questions. Tick or circle

• Open ended:

• - Record responses during interview

• - Use the respondent’s own words. Verbatim

• - Do not summarize or paraphrase.

• - Include everything that pertain to question objectives.

• - Include all your probes.

Terminating the interview• No hasty departure. Secure information.

• Respondent’s spontaneous comments at the end.

• Answer any question by the respondents.

• Leave by observing local customs. “Don’t burn your bridges”

• Thank for the cooperation.

• Find a place to edit. Record information on face page.

FOCUS GROUP DISCUSSION

• FGD are a more formal way of getting groups of people to discuss selected issues.

The purpose of FGD:

• To obtain in-depth information on concepts, perceptions, and ideas of the group.

• More than an question-answer interaction.

• Group members discuss the topic among themselves.

Focus groups are:

• Formally constituted (organized in advance);

• Structured groups brought together (people from similar background, age, sex, education, religion, or similar experiences);

• 6-12 persons;• Guided by a moderator/facilitator;

Cont.

• To address a specific issue (talk freely, agree or disagree),

• Within a fixed time frame, and • In accordance with clearly spelled

out rules of procedure.

How to conduct FGD?

1. Preparation

• Selection of topic, questions to be discussed (open ended).

• Selecting the study participants:• - Purposive or convenience sampling. • - Similar background. Age, sex, status.• - 6-12 persons in a session.• - Contact the participants. Invitations.• Making physical arrangements.

2. Conducting the session:

• One of the members of the research team should act as ‘facilitator’ for the focus group. One should serve as ‘recorder’ (rapporteur).

Functions of the Facilitator/Moderator:

• Act as a coordinator, not as an expert.• Introduce the session.• Encourage discussion.• Encourage involvement. Ask for

clarifications; reorienting the discussion when it goes off the track; bringing in reluctant participants. Deal with dominant participant

• Build Rapport. Observe nonverbal communication.

Some guidelines for the facilitator:

• Observe verbal as well non-verbal communication. What are they saying? What does it mean to them? Empathize.

• Do not try to comment on everything that is said

• Control the timings unobtrusively.

Some guidelines for the Facilitator (Cont.)..

• Summarize the main issues at the end of FGD.

• Check whether all participants agree.

• Thank the participants.

• Listen for the added comments after meeting has closed.

Functions of the Recorder

• Keep a record of the content as well as emotional reactions and nature of group interactions.

• Record the following:

• - Date, time, and place.

• - Names and characteristics of participants.

• - Description of group dynamics (level of participation, presence of a dominant participant, level of interest).

• - Opinions of the participants, as far as possible in their own words, especially the

key statements.

• - Emotional aspects (reluctance, strong feelings attached to certain opinions)

• - Vocabulary used.

Functions of Recorder (Cont.)

Functions of Recorder (Cont.)

• Assist the facilitator by drawing his/her attention to missed topics or missed comments from the participants.

• Help the facilitator resolve the conflict if necessary.

• Make sure a copy of the list of topics and key probe questions is available and referred to during the FGD.

Duration of FGD

• Between one hour to one hour and a half.

3. Analysis of Results

• After each FGD the facilitator and recorder meet to review and evaluate the discussion.

• The full report is prepared using the participants own words, listing the key statements, ideas and attitudes.

• Additional questions are formulated if needed.• Answers of different sub-groups are compared.• The findings must be recorded in a coherent

way.• The most useful quotations should be selected.

4. Report Writing

• Start with a description of the selection and composition of the group and participants, and a commentary on the group process.

• Present your findings, following your list of topics and guided by the objectives of FGD.

• Include questions whenever possible, particularly the key statements.

Limitations

• Results cannot usually be used for generalization.

• Participants often agree with responses from fellow members (for different reasons). Researcher to be cautious when interpreting the results.

•

Cont.

• The moderator may influence the participants (bias).

• FGDs have limited value in exploring complex beliefs of individuals.

• FGDs can paint a picture of what is socially acceptable in a community rather than what is actually occurring or believed. (real and ideal problems)