EFFECTIVENESS AND EFFICIENCY OF ENVIRONMENT AWARENESS

RAISING PROJECTS

Ádám JüttnerTNS Hoffmann Ltd.

The FÉF (Sustainable Lifestyle and Consumption) projects aim to change social norms and behaviour at the same time

• The assessment examined the implementation, effectiveness and efficiency of the projects

• The aim of EEOP 6.1.0 and 6.2.0 Sustainable Lifestyle and Consumption constructs was:– informing the public, raising awareness– developing environment consciousness and

environmental culture (attitude shaping)– propagating behaviour patterns related to

sustainable lifestyle

• The targets of the result indicator were campaign reach scores

Large number of projects and significant absorption in the EEOP’s priority aiming to shaping attitude

• The starting point of this survey was a database, containing the data of altogether 796 contracted FÉF projects that received funding from EEOP between 2008 and October 2012 *

• Based on the contracts, funding amounting to nearly 12 billion HUF was used to implement 408 campaigns and 388 pilot projects.

3-5 projekt

6-10 projekt

10-nél több projekt

1-2 projekt

0 projekt

Number of projects per microregions between 2008-2011 (contracted projects)

0 project

1-2 projects

3-5 projects

6-10 projects

10+ projects

It is indisputable that EEOP FÉF campaigns fill an important absence

• Except for the 408 FÉF campaigns, there were altogether 193 non EU-funded campaigns related to sustainability.

• The issue of sustainability was mainly kept on the agenda from EU funding

Communication related to sustainability

Printed press

Online (press/webpages

)

Outdoor posters

TV /radio

EEOP FÉF campaign elements (number)

339 292 174 229

Number of campaigns not funded by EEOP*

136 38 28 50

*Source: IMEDIA, 2008-2011 Campaigns related to environmental protection, waste management and sustainability; private businesses for the most part, we have no information about the funding

The targets of the priority result indicator were met

• Based on the Feasibility Study plan data, the campaign reach targets were met every year (well above the set targets!)

• Consequently, the campaigns and pilot projects reached a wider range of people than initially planned.

• OP target indicators initially were underestimated, these have been raised in 2012

Name of indicatorIndicator

unit

2008 2009 2010 2011

EEOP target

FS Plan data

EEOP target

FS Plan data

EEOP target

FS Plan data

EEOP target

FS Plan data

Campaign reach indicators based on activity type

Short time /Passive participation*

thousand people 7 000 30 317 14 000 77 596 21 000 645 400 35 000 425 279

Long time/ Passive participation*

thousand people 2 000 7 892 4 000 256 129 6 000 35 877 10 000 110 222

Short time/ Active participation

thousand people 12 208 24 295 36 557 60 246

Long time/ Active participation

thousand people 1 377 2 152 3 552 5 204

(* A passzív elérési mutatók kumulatív összegek: egy ember annyiszor szerepel a statisztikában, ahány alkalommal találkozott a „Fenntartható életmód és fogyasztás” prioritási tengely keretében támogatott bármely kommunikációs tevékenységgel)

Based on Eurobarometer data, change is perceptible both as regards awareness and behaviour across the entire Hungarian society

Awareness:•People have become perceptibly better informed about environmental issues (in 2007, 41%, whereas in 2011, 55% were somewhat informed about this issue).

Propagation of behaviour patterns:•Between 2007 and 2011, the number of people who reduced their energy consumption grew significantly (from 41% to 63%) • The number of people who claim that they collect waste

selectively (from 49% to 54%) and those who reduced their water consumption (from 29% to 48%) also went up

EFFECTIVENESS OF AWARENESS RAISING PROJECTS

Cost of reaching

There are significant differences between the efficiency of the projects (based on FS data)

Code Supported activity

Average reaching of

target group (number of

people)

Cost of reaching a

person (HUF)

EEOP-6.1.0 Awareness-raising campaigns 3 566 669 4,8

/A local campaign 7 531 598,9

/B regional campaign 7 269 683 4,7

/C national complex campaigns 76 242 846 1,9

EEOP-6.2.0 Sustainability pilot projects 148 177 681 38,3

/A local pilot projects 99 795 86,5

/B regional pilot projects 4 863 142 22,6

Total 2 010 958 7,9

TOP2 regions with highest active reach percentage - regional projects

County / microregion

Percentage of active reach

(active reach/ population of county)

Number of projects

Regional campaign (EEOP

6.1.0/B)

Regional pilot project

(EEOP 6.2.0/B)

Tolna county: Szekszárd microregion 95% 3 1

Hajdú-Bihar county: Hajdúszoboszló microregion

50% 2 0

TOP2 regions with highest active reach percentage – local projects

County / microregion

Percentage of active reach

(active reach/ population of microregion)

Number of projects

Local campaign (EEOP 6.1.0/A)

Local pilot projects (EEOP 6.2.0/A)

Csongrád county: Mórahalom microregion 38% 3 5

Somogy county: Lengyeltót microregion 37% 1 8

Based on the plan data of the FS report.

Regional projects had the highest planned reach percentages

RECOMMENDATION: Benchmark for required level of efficiency

• When assessing individual projects, benchmarks should be identified for the particular project type.

• Based on the benchmark data, a required level can be identified. Projects that are over the benchmark can be considered efficient, while those below the benchmark would have to be examined:– E.g. in the case of school projects

• In short-term active reach, the top 30% reached over 230 people, while the bottom 30% reached fewer than 70 people.

• Therefore, in the case of short-term active reach, between 0-70 people the project should be examined, between 71-230 people it is adequate and over 230 people, efficient.

RECOMMENDATION A benchmark database should be compiled based on the efficiency indicators

• A continuously expanded benchmark database would provide an objective measure for assessing projects.

• Some actual data should be systematically added to the existing project database, and the key efficiency indicators should be identified on the basis of those– Active reach /number of people at the project location – The cost of reaching one person (project cost/ active

reach)• Good and bad practices could be identified with the help

of the indicators (e.g. TOP10% or TOP30%).• The database could be used as a point of reference when

assessing future projects.

• The projects that have the highest reach percentages and lowest reach/person costs could be presented to projects hosts on a regular basis, either on county or micro-regional level or based on project types.

• Best practices could become examples to follow if the factors contributing to its success are analysed and project hosts are provided an opportunity to learn from them

• In addition to best practices, some conclusions should be drawn from negative examples, too, identifying ‘bad practices’ that should be avoided during future projects.

• All this could contribute to more efficient and successful implementation.

RECOMMENDATION: Feedback to project hosts

EFFICIENCY

raising awarenessdeveloping environment

consciousness propagating behaviour patterns

Measuring the effect of EEOP awareness raising projects – Desk research and measuring of effectiveness by using ‘contrafactual’ method

• Desk research: measuring the effectiveness of regional campaigns

• Effectiveness assessments carried out by project hosts have been assessed

• In the case of local campaigns and pilot projects, we measured effectiveness by using ‘contrafactual’ method:– School programmes

(altogether 146, total budget 647 million HUF)Based on OP indicator: active reach

– Pilot projects promoting composting (altogether 231, budget nearly 2 billion HUF)

Based on OP indicator: output indicators verifying the progress of activities undertaken in the project proposal

School programmes were the most successful in expanding the target group’s knowledge

• In the case of school projects, we detected changes mainly in the schoolchildren’s knowledge:

– they know more about environmental problems

– project participants are more aware of the harmful effects on human activities on the environment

– of environment-conscious behaviour patterns, a positive change in behaviour can be detected as regards selective waste collection

In the case of pilot projects promoting composting, we can detect a clear change in behaviour

• The pilot projects promoting composting mainly influence behaviour:

– project participants thought that getting a composting box was useful; the project was well-received

– in settlements participating in the projects, there is a higher percentage of people who compost waste

– according to feedback from participants, composting has become a part of their weekly routine and they have noticed a reduction in the amount of household waste

The current practice of measuring effectiveness by project hosts

• In most cases, effectiveness was measured with the help of questionnaires, in most cases in paper-based, self-completion format, and in some cases, using face-to-face interviews (carried out by interviewers).

• In most cases, effectiveness was assessed by measuring it twice (typically, at the beginning and end of the project).

• When assessing effectiveness, they mainly measured knowledge and attitudes (in 43% of cases), while campaigns were evaluated in 20% of projects.

• The effectiveness of projects were measured by the applicants themselves the most frequently (41%). 

RECOMMENDATION: Mandatory measuring of effectiveness, containing standardised sections 1.

• Out of 796 projects, it was mandatory to measure effectiveness in the case of 387 projects, which was carried out individually by the participants, using different methods and often, with different aims, so it is not possible to compare the results.

• We suggest that in the future, all participants of such projects should be required to measure effectiveness:– A professional, methodological component should be

drawn up, which would be standardised for all projects and would constitute a part of individual assessments of effectiveness.

– The remaining part of the assessment should be tailored to the programmes, which are homogeneous as regards their theme and target group, but should also be standardised for the particular programme type.

RECOMMENDATION: Mandatory, standardised measuring of effectiveness 2.

• It should be adapted to current practices (wording and structure optimised for self-completion questionnaires, standardised set of questions).

• It could be extended onto all project types.• It should be suitable for use before starting the project,

after completion and as follow-up. • It should address all of these points:

– what is the target group’s current level of knowledge about sustainable lifestyle and environment-consciousness

– what are the target group’s attitudes to sustainable lifestyle and environment-consciousness

– how these attitudes have been integrated into the target group’s day-to-day behaviour

Conclusions - efficiency

• Compiling a benchmark database could be helpful in assessing future projects and in identifying projects that are more or less efficient than the average.

• By presenting best and worst practices, applicants would be provided with some assessment criteria, which could contribute to improving the efficiency of projects.

• It would become possible to identify the most efficient types of campaigns and pilot projects, which could result in improved allocation of resources (naturally, effectiveness targets should also be taken into consideration here!).

The effect of EEOP awareness raising projects

• Effectiveness does not automatically mean that the projects are efficient.

• Based on international publications, campaigns and pilot projects can mutually contribute to each other’s success:– until sustainable behaviour patterns become sufficiently

widespread, it is important to keep environmental issues on the agenda with the help of campaigns

– communication plays an important role in preparing participation in pilot projects, i.e. people should be made interested and a positive attitude should be created first

• To be able to compare and assess such projects, it would be necessary to incorporate some partially standardised measuring tools and quality assurance criteria into measuring effectiveness, which is currently carried out individually.

Thank you for your attention!