kap surveys

KAP SURVEYS

Purpose of KAP survey• Media market • Trusted sources• Ground truth assumptions • Barriers/Benefits to behavior change.• Segment campaign TA • Stages of Behavior Change continuum

Purpose of KAP survey• Baseline - SMART objectives• Assess campaign• Next steps.• Comparability of data set• Exposure • Attribute impact to Pride campaign.

Where do the results go?• Results section

Appendix H4 (p39-83 Lola)

• KAP section of Monitoring plan (SMART objectives) Monitoring plan excel spreadsheet

• Audience profile Understanding our audience B5 (see template plan)

• Flagship species & local ambassadorExecutive summary (A2)

• Public consultation

SEQUENCE OF KAP ANALYSIS

1. Segmenting the audience and analyzing data in Survey pro

2. Summarizing data in results section per TA

3. Filling in monitoring plan for KAP & setting SMART objectives

4. Summarizing strategic data in audience profile

5. Selecting data and format and presenting at public consultation

6. Flagship sp & Ambassador to exec. summary

What should the results section cover?

• Independent variables• Trusted sources and media channels• Stages of behavior• Knowledge • Attitude & self efficacy • Interpersonal Communication• Behavior and practice• Barriers and benefits to behavior change• Exposure• Ground truthing (incl. flagship sp & climate change)

• Tables• Insert graphs

remember to label them • Do not just follow template

base summary on your survey and data

• Highlight interesting results

How to summarize results ?

(p39-83 Lola)

Example of summary in table (p39-83 Lola)

Information Source Level of

Trustworthiness TA1 Fishers

(N = 120)

TA2 Community (N = 2000)

TA3 (N= xxx)

Person on radio

Most 0% 0% 3% Very 17% 30% 18% Somewhat 83% 50% 68% Not 0% 0% 5%

Person on TV


Report in newspaper or magazine


Law enforcement official


Celebrity/Soap Star/Entertainer Preferences

It was apparent that Robin Padilla was the most popular with 34% of the responses.

Example of how to highlight a result

KAP section of Monitoring Plan

KAP survey Monitoring

Monitoring BR BC TR

Monitoring CR Cohort ToC

KAP section of Monitoring plan

Results chain Survey question

Reduced fishing in Tandayag MPA and Buffer Zone

(Q53, A) I am going to read you a list of different types of fishers, and for each one, I would like you to tell me whether you remember seeing someone like that fishing in this area in the past 6 months (A) Subsistence fishers from your village [ ] Seen


Pre campaign (baseline) Target Desired pp change (specify if increase or decrease)

40% of respondents say they have seen a subsistence fisher from the village fishing inside area A (the MPA)

20% 20pp decrease

Baseline & Target


(TR) Threat Recution The % of respondents who say they have seen a fisher from the village fishing within area A (MPA) will decrease from 40% measured in March 2010 to 20% in August 2012 (a decrease of 20 pp, Q53 in KAP survey)

SMART GoalsExample: The % of respondents who say... will increase/decrease from XX% measured in March 2010 to YY% measured in August 2012 (an increase/decrease of YY-XXpp, Qx in KAP survey)


Pre campaign (baseline)

Desired pp change

Post campaign result

Attained pp change

Measuring impact

Audience Profile• In General / What do we know about this group?• Knowledge• Attitude• Practice• Trusted sources• Media sources• Any other relevant information• Comments

(template plan & p84-86 Lola)

Example from audience profile

Trusted sources fishers• The most trustworthy groups for

Fishers are the Religious Leaders, Friends or Family.

• All officials are not thought of as particularly trustworthy for environmental information.

• TV, Radio and Newspapers, Magazine are only considered “Somewhat trustworthy”.

Example from audience profile

Media preferences fishers• Radio: 33% of Fishers never listen to

the radio, 33% listen up to 3 days a week, the favorite station is DYRD, and the most popular time is before 6.00am or between 6.01am – 10.00am.

• Fishers said that Tagalog Love Melodies are their most liked (25%) media type. 58% did not like Dramas, 83% did not like Puppet Shows and 67% did not like Locally produced videos.

Flagship species and ambassador

Executive summary• Flagship species common name• Flagship species scientific name• Flagship species details (<200 words)

Brief description and why it was chosen

• Campaign ambassadorBrief description of who the person is and why they were chosen

• Cohort ambassador

Purpose of public consultation

Crissy’s session

EXERCISE Selecting KAP data for public consultation

• Split into groups of 4

(10 Min)• Browse through results and audience profile in Lola (p 49-86) • Underline any data that you think is particularly interesting(5 Min)• Decide as a team on 5 KAP results that you think would be

strategic to present at a public consultation and why (1 min)• Share with everyone three results you want to present and

why

Setting SMART objectives

Purpose of SMART objectives

• Focus• Motivation• Assess and demonstrate success• Adaptive management

% or pp ?

Pre campaign Post campaign Pp change % change20% fishers 39% fishers 19pp increase Increase of 95%

(19*100)/2020% gleaners 18% gleaners 2pp decrease Decrease of 10%

(2*100)/20

Fishers who say the MPA size is “just right”

The potential for change for different types of objectives across the Theory of Change

26

Criteria K A IC BC

Average Percent of Target Audience Changed

22pp 13pp 28pp 14pp

Sample Size (n of campaigns) 213 139 42 45

It is easier to change knowledge and generate conversations than to change attitudes and behaviors.

Historic Pride campaign average results – summary till 2010

Module 2, Unit 5, Session 2

SMART Objectives: What is Realistic?

Criteria K A IC BC

Average Percent of Target Audience Changed 22pp 13pp 33pp 10pp

Average Number of People Changed 9,490 7,586 15,128 2,572

The potential of change for different types of audiences

28

Baseline\Target Audience (data based on median of knowledge)

General Public

Influencer Resource User

<20% 20pp 24pp 1.7pp20% to 40% 23pp 37pp 25pp40% to 60% 31pp 33pp 21pp>60% 17pp 8pp 16pp

Selective perception (Hassinger) plays a role – people who don’t “want” to know somehow don’t seem to learn.

• Selective perception• The critical mass phenomenon / social norms

The potential of change for different baselines

29

According to Diffusions of Innovation the Rate of Change Depends on the Starting Point

0

10

20

30

40

50

60

70

80

90

100

0 5 10 15 20 25 30

Time (Relative)

% A

dopt

ion

of In

nova

tionn

Innovators – 2.5%

Early Adopters – 13.5%

The Late Majority – 34%

Laggards – 16%

The Early Majority – 34%

Source: Everett Rogers, graph from Wikipedia.org

PEP1 results – Madagascar

Category

Average % of Objective Achieved

Number of people changed (max per category)

Average percentage point change

Knowledge 79%

180 19.0

Attitude 57%

100 9.5Interpersonal

Communication 92%

300 40.5Barrier Removal

Behavior Change 57%

170 25.4Threat Reduction 85%

Conservation Result

Target Audience Name

Target Audience Size

General Public 7,500Local Leaders 70Pirogue Owners

1,000

Beach Seiners 30

K 22pp

A 13pp

IC 28pp

BC

14pp

By October 2010K – 89% of pirogue owners are aware of local Dina (up from 71%

in March 2009; 18pp increase)

A – 97% of local leaders believe they are the ones responsible for Dina enforcement (up from 88% in March 2009; 9pp increase)

IC – 83% of local leaders have talked to someone about no longer using beach seine fishing (up from 33% in March 2009; 50pp increase)

By October 2010IC – 43% of pirogue owners have talked to someone about no

longer using beach seine fishing (up from 13% in March 2009; 30pp increase)

BC - 28% of pirogue owners have warned any lawbreakers with respect to poison or seine fishing (up from 11% in March 2009; 17pp increase)

BC - 48% of leaders who say they have helped in Dina enforcement (up from 11% in March 2009; 27pp increase)

BC – The number of fishermen who beach seine has gone down from 26 to 18.

Some BR results• 28 cases of dina infraction reported• 7 cases fined (1 fined by the Mayor)• 20 cases discussed and warned• 8 beach seine net owners have left their

net or changed their methods (including the Mayor)

• Involvement of fisheries department and the Mayor

PEP1 results – Guam

Category

Average % of

Objective Achieved

Number of people

changed (max per category)

Average percentage

point change

Knowledge 262%

11,426 36.5

Attitude 71%

3,116 14.5Interpersonal

Communication 172%

9,868 18.0Barrier Removal 63%

Behavior Change 166%

16,101 19.5Threat Reduction

Conservation Result



General Public 51,938Hunters 600

K 22pp

A 13pp

IC 28pp

BC

14pp

PEP1 results – Fiji

Category

Average % of

Objective Achieved

Average percentage

point change

Knowledge -58% 5.5Attitude -227% 3.0

Interpersonal Communication -17% 8.9Barrier Removal 65% 65.0Behavior Change 10% 9.7Threat Reduction

Conservation Result

Target Audience Name Target Audience Size

General Public 14,000Chiefs Fishermen

K 22pp

A 13ppIC 28pp

BC 14pp

PEP1 results - Thailand

Category




Knowledge 87%

12,145 17.6

Attitude 95%

15,615 20.4Interpersonal Communication 75%

14,459 22.8

Barrier Removal 106% 80.5


20,242 28.8Threat Reduction 60% 6.0Conservation Result



General Public 57,834Sellers 200Government Officers

K 22pp

A 13pp

IC 28pp

BC 14pp

PEP1 results - Lao

Category




Knowledge 88%

4,776 11.6

Attitude 98%


2,212 17.3

Barrier Removal 73%

2,630 10.1

Behavior Change 18%

495 6.8Threat Reduction Conservation Result



General Public 11005Illegal hunters (village militia & police) 190Government Officers (army 100, police 20, patrol team) 126

K 22pp

A 13pp

IC 28pp

BC 14pp

PEP1 results – Mongolia WWF

Category




Knowledge 104%

800 21.5

Attitude 356%


1,500 32.3

Barrier Removal 109%

640 32.0


894 34.7Threat Reduction Conservation Result 600%?



Local fishers 2,000

K 22pp

A 13pp

IC 28pp

BC 14pp

Penablanca Philippines campaign

Target Audience

NameTarget

Audience Size

General public

26,308

Farmers 12,825Fishermen 53

Students 17+ 1,688

Category% of Goal Achieved

Number of people

changed (max per category)

Average percentage

point change

Knowledge 116%

15,522 27.3

Attitude 184%

6,840 36.3Interpersonal

Communication

Barrier Removal

Behavior Change 79%

6,577 16.3Threat Reduction 100% 5.0

Conservation Result

K 22pp

A 13pp

IC 28pp

BC 14pp

Historic data summary

2005 2006 2007 2008 20090

10,000

20,000

30,000

40,000

50,000

60,000

Average number of people changed per campaign each year

KnowledgeAttitudeInterpersonal CommunicationBehavior Change

Cohort ToC targets• K - By Year 1 the percentage of the population aware and

clear on the benefits of the MPA is greater than>55% in the village nearest to the MPA and greater than 25% for the whole town

• K - By Year 2 indicator 1 of the Eco Gov CCEF MPA rating system (MPA concept explained and accepted)

• K - By Year 3, 90% of village and 60% of the town are aware of the MPA and its benefits

Cohort ToC targetsA - Attitudes and knowledge towards municipal MPAs shift

towards supporting the MPA and viewing it as an asset as measured by community surveys:

• by year 1, 60 % • by year 2, 90 % • By year 3 significant pride in the MPA is generated and it is

seen as an integral part of the community

Cohort ToC targetsBR/BC• Enforcement of reported incursions into the MPA increase to 90%

within 2 years 8.• Local community members join in (a) annual monitoring surveys

which track trends within the MPAs and (b) management committee general assemblies

• Achieve indicators 13 and 24 of the Eco Gov CCEF MPA rating system within 2 years. (Regular participatory biophysical monitoring being conducted; Enforcement activities initiated)

BC • Reported intrusions from intelligence information passing to the

management committee increase by 60% by year 2

Cohort ToC targetsTR • The majority (target: >80%) of continued

intrusions after year 1 come from outsider fishers (not from local community)

• 90% decrease in intrusions from adjacent village community members into the MPA by year 2

• 90% decrease in illegal and destructive fishing within 500 meters from MPA boundaries by year 2 and 99% reduction by year 5

Setting SMART objectives

•Inspiring•Realistic

KAP Survey TOP TIPS Planning & rolling out survey• Community Based Monitoring System (CBMS) map ; from MPDO• Shortening surveys: check survey repetition & which questions to ask TA • Work with Barangay captains

Enumerator training• Motivation trip • Community enumerators• Have an ID

“We are making history”

Our opinion matters”

KAP Survey TOP TIPS Encoding• Dry run

• Change the structure of the open ended questions before encoding

• Encoders who are willing to work evening/night shifts. Don’t try and do all the encoding yourself: get some help!

• Be strategic about the order in which you encode (and also survey) the target audiences start by encoding fishers and the local community (TA1 &TA2)

• Write the corresponding form number in Survey pro on the paper survey

Exercises

SEQUENCE OF KAP ANALYSIS

1. Segmenting the audience and analyzing data in Survey pro

2. Summarizing data in results section per TA

3. Filling in monitoring plan for KAP & setting SMART objectives

4. Summarizing strategic data in audience profile

5. Selecting data and format and presenting at public consultation

EXERCISESSegmenting the audience and analyzing data in Survey pro1. Cross tabulation2. Rescale answer options3. Creating a filter4. Exporting a graph into word5. Analyzing open ended questions

Practice/Questions6. Filling in KAP monitoring plan

Filling in Results sectionFilling in audience profile

Using your encoded Survey Pro fileFind out the baseline data for all of your TA to the question:

(30) What do you think of the size of the MPA? [ ] Too Big [ ] Too Small [ ] Just the right size[ ] Unsure [ ] N/A(Q30/107 cohort template; Q34/107 Vince; Q38/116 Anabele)

To do this cross tab the above question with the TA question (Q1 in your survey)

1.Cross tabulation

Vince’s survey, Q34/107 vrs TA question (Q1/9)

Audience segmentation using TA question

Audience segmentation using TA questionVince’s survey, Q34/107 vrs TA question (Q1/9)

Audience segmentation using methodology questionFrom Anabele’s survey Q38/116 - Q1/Q8

2. Rescale answer options

Combine two or more answer options to streamline segmentation

2. Rescale for answer optionsAnabele’s survey, Q38 /116 - using TA Q1/Q8 combining all categories into one TA: Fishers&Gleaners Tandayag

Which of the following statements best describes you [choose only ONE]: -You are a fisher that fished at least 1 time in the past month near the Tandayag Marine Sanctuary, - You are a fisher that fished at least 1 time in the past month in the Tandayag Marine Sanctuary, - You are a gleaner that gleaned at least 1 time in the past month near the Tandayag Marine Sanctuary, - You are a gleaner that gleaner at least 1 time in the past month in the Tandayag Marine Sanctuary,

2. Rescale for answer optionsAnabele’s survey, Q38 /116 - using TA Q1/Q8 combining all categories into Fishers/Gleaners Tandayag

Using your encoded Survey Pro fileFind out the baseline to the question (30) What do you think of the size of the Lola Marine Sanctuary? for people whose primary or secondary occupation is fishing

To do this create a filter with 2 questions on primary and secondary occupation

3. Creating a filter

What is your primary occupation or sector in which you work? Q7/15 Anabele

What is your secondary occupation or sector in which you work? Q8/16 Anabele

3. Creating a filter

4. Export a graph into wordCustom report: Select table/figure and right click “copy” and then in word “paste special” (Device unformatted bitmap)

Executive Summary:Document => Publish Report

As web Html Þopen the file with word => save as word

As Printable images=> Saves in folder with=> Import individual images into word

Options 1Clean up categories for things people talked about in relation to “ benefits to the community of a well enforced sanctuary” (Q41A/131 Vince’s survey)

To do this use the “Answer replace” functionDatabase => Answer replacereplace “about fishings” with “about fishing”

5. Analyzing open ended questions

Option 2: Before starting encoding you could:Create categories for things people talked about in relation to “ benefits to the community of a well enforced sanctuary” (Q41A Vince’s survey – Q37A in Cohort template)

To do this browse through answer options in paper surveys and select the most common answers, then change the structure of question and include a “blank” answer option

-Fishing-Community support -Boundaries-Rules and regulations-Other______-N/A


Option 3 - Using your encoded Survey Pro fileAdd one or more common answer categories

What local community benefits are there from having the no-take area/core zone nearby? You can tell me up to 3 that you know. (COHORT QUESTION, OPEN ENDED) (Q23B in Cohort template)

To do this browse through answer options and select the most common answers, then add these to the answer options


Example:In the past 6 months, have you talked to anyone about benefits to the community of a 24/7 guard system of the Tandayag/Bio-os marine sanctuary? If you have, with whom have you talked to about this. Q 46 Anabele’s survey

If you did talk about this, can you tell me what was the main thing you discussed?Rules and regulations of the MPAEquip fixture of guardhouseInvite others to visit the MPAConditions of guard houseImportance of ensuring MPA protection

=> Add category: More fish if we protect

Filling in monitoring plan from KAP

•Is the baseline data suited & strategic for setting a SMART objective?

•What answer options (or combination of) do you want to use as your target?

•What is a realistic target?

A SMART Objective for Every Step in the Results Chain and Every Audience

Including BC, BR, TR & CR

Module 2, Unit 6, Session 3

Exercise on KAP monitoring plan

Find out what your baseline data for the following cohort question is:

I am going to read you a list of different types of fishers, and for each one, I would like you to tell me whether you remember seeing someone like that fishing in this area in the past 6 months (show an area that is a NTZ on a map of the area but don’t mention whether it is NTZ or not) (Q46/151-161 Anabele)

Then set a target and enter it in the monitoring plan

For example

Pre campaign 40% of respondents have seen subsistence fishers from the Barangay fish inside area A (NTZ of MPA)Target could be to reduce this to 10% (30pp decrease)

ANY QUESTIONS?PAOTANA?

kap surveys

Documents

results section cover

audience b5

spreadsheet audience

lola example of summary

strategic data

survey pro summarizing

template base summary

trustworthy groups