WORKSHOP HANDOUT

Designing capacity-building interventions: what the evidence tells us

Prepared by the Health Promotion and Special Projects team, Health Promotion and Chronic Disease

Prevention Department at Public Health Ontario. Contact Benjamin.rempel@oahpp.ca for details.

Introduction Public Health Ontario recently conducted two systematic reviews to determine the effectiveness of

capacity building interventions relevant to health promotion and public health. The purpose of this

workshop is to translate the knowledge gained from these reviews and apply it to the interventions used

in participants’ own practice.

What is capacity building? The World Health Organization defines capacity building as “the development of knowledge, skills,

commitment, structures, systems, and leadership to enable effective health promotion, with actions to

improve health.”1(p.341-342) The definition includes action at three levels:

1. The advancement of knowledge and skills among practitioners

2. The expansion of support and infrastructure for health promotion in organizations

3. The development of cohesiveness and partnership for health in communities1

PHO conducted two systematic reviews to examine the evidence around the role of capacity building in

public health. Two articles summarizing their results were published in BMC Public Health: Theories,

models and frameworks used in capacity building interventions relevant to public health;2 and

Effectiveness of capacity building interventions relevant to public health practice.3

Theories, models and frameworks This systematic review2 was conducted to identify conceptual foundations for capacity building by

examining the theories, models and frameworks used to support capacity building. Nineteen papers

were included in the review: eight were rated as strong and 11 papers were rated moderate, primarily

as the theory, model or framework was implicitly rather than explicitly stated.

Using theories, models or frameworks as a foundation for developing capacity building interventions can

provide a road map for studying programs, developing appropriate interventions, and evaluating their

effectiveness.2 A good theory provides a clear explanation of how and why specific relationships lead to

Workshop Handout: Designing evidence based capacity building interventions that work for public health The Ontario Public Health Convention, 2019 Page 2

Adapted from McLeroy K, Steckler A, Bibeau D. The social ecology of health

promotion interventions. Health Education Quarterly, 1988 15(4):351-377.6

Societal

Community

Organizational

Interpersonal

Individual

specific events2 and how individuals, groups and organizations behave and change.4 Using this

knowledge can help us to design programs and to:

build understanding of our stakeholders

explore the factors that can promote or inhibit change

help us to select and develop health promotion strategies2

In other words, theories can help us to understand who we should be targeting, why certain

individuals/groups engage in certain behaviours, and how we can achieve change through our

programming.4 Using theories, models and frameworks to design our capacity building interventions can

help us to:

describe/guide the process of translating research into practice

understand and/or explain what influences implementation outcomes

evaluate implementation5

Four theories, 17 models, and seven frameworks were identified in the studies included in the

systematic review. Five were identified for consideration as a theoretical foundation for designing

and implement capacity building approaches relevant to public health. Each is described below:

DIFFUSION OF INNOVATIONS

Diffusion of Innovations is a behaviour change theory which addresses how ideas, products, and social

practices that are perceived as new spread throughout a society, or from one society to another.4 A

number of factors determine how quickly, and to what extent, an innovation will be adopted and

diffused:

The relative advantage of an innovation shows its superiority over whatever it replaces.

Compatibility is an appropriate fit with the intended audience.

Complexity has to do with how easy it is to implement the innovation or adopt the behaviour

Trialability refers to whether it can be tried on an experimental basis.

Observability reflects whether the innovation will produce tangible results.4

ECOLOGICAL FRAMEWORK

The Ecological Framework focuses on the complex

interactions between people, groups and environments.6

This framework recognizes that each of us are influenced

by the environments in which we live, and that each of the

factors in the framework shape the choices that we make.6

At the most basic level, an ecological perspective points to

two approaches addressing health problems: change people’s

behaviour, or change the environment. There is a third and

more powerful option, which is to integrate these two

Workshop Handout: Designing evidence based capacity building interventions that work for public health The Ontario Public Health Convention, 2019 Page 3

Adapted from: Wandersman A. et al. Bridging the gap between prevention research and

practice: The interactive systems framework for dissemination and implementation.

American Journal of Community Psychology, 2008;41(3-4); 171-81.11

approaches and treat them both as essential.4 This option recognizes reciprocal causation: that

individual behaviour both influences, and is influenced by the environment.4

TRANSFORMATIONAL LEARNING

Transformational Learning Theory is a behaviour change theory2 that uses “disorienting dilemmas” to promote growth and change. Individuals are challenged to assess how a learning experience interacts with their existing value system and worldview. Successful learning experiences result in individuals who are in some way (unique to each individual) transformed by an experience.7

INTERACTIVE SYSTEMS FRAMEWORK FOR DISSEMINATION AND IMPLEMENTATION

This framework seeks to address the “how to”

gap between science and practice.8 It was

developed in the violence prevention sector,

where despite the evidence of what works and

what does not, there has not been wide spread

of effective strategies.8 The framework shows

how different systems can work together for

successful dissemination and implementation

of prevention innovations. Each system

involves many different people in different

roles, different stakeholders and organizations,

with all three working together to:

Distill the scientific literature into

findings that are disseminated to the

field

Support the work of capacity building,

training and technical assistance

Deliver and evaluate prevention

strategies.8

BLOOM’S TAXONOMY OF LEARNING

This is one of the most recognized

learning theories in the field of

education.9 Bloom’s Taxonomy can be

used to create learning outcomes that

target not only subject matter but also the

depth of learning that is desired.10 The

taxonomy contains a 6-part hierarchy:

remember, understand, apply, analyze,

evaluate and create.

Workshop Handout: Designing evidence based capacity building interventions that work for public health The Ontario Public Health Convention, 2019 Page 4

Intervention Type

Internet-based instruction

Training and workshops

Technical assistance

Education using self-directed learning

Communities of practice

Multi-strategy interventions

Outcomes

Knowledge

Skills

Self-efficacy

Change in practice and policy

Behaviour change

Application

Organizational support

Perceptions of system-level capacity

Effectiveness of capacity building interventions This systematic review assessed the effectiveness of capacity-building interventions relevant to public

health practice. Thirty-eight papers were assessed as relevant, and 24 of those were excluded due to the

low quality of the study. The remaining 14 were included.

The impact of 6 capacity-building interventions on 8

outcomes were assessed:

Internet-based instruction is defined as any computer assisted instruction using the internet as the

mode of delivery. Internet-based instruction increased knowledge and skills, but only in comparison to

no other intervention. Including discussion groups in the intervention improved knowledge, while

including practice sessions improved skills. Level of interactivity and repetition made no difference for

knowledge or skill enhancement. Recommendation: use when no other interventions are feasible or

accessible (for example due to limited resources or geography/reach).

Training and workshops are defined as in-person training and workshop sessions. These interventions

were grouped together as it was difficult to clarify the difference between training and workshops.

Training and workshops resulted in enhancements across all outcomes with the exception of

understanding, though this is likely because of differences of groups at baseline in the study. Not all

studies examined impacts on behaviour, improvement in practice/policy, and application of knowledge.

Recommendation: closely read the studies that showed effects on behaviour change, improvements in

practice and policy, and application or knowledge in order to identify the key elements of the

interventions that results in these outcomes.

Technical assistance includes personalized support, including face-to-face meetings. The included

interventions reported changes in knowledge, skills, leadership and self-efficacy. The results for changes in

practice and organizational support were mixed. Recommendation: use technical assistance to

complement training and workshops. Evaluate baseline conditions and consider a feasible level of change.

Self-directed learning is any self-directed methodology that includes: facilitators (versus teachers);

learners involved in in selecting learning resources; and self-assessment. This intervention showed some

impact on knowledge when learners are more involved in learning resource selection and for those who

are more advance in learning. Interactivity of learning modules did not make a significant difference in

outcomes. There is a moderate increase in knowledge compared to traditional teaching methods.

Recommendation: build on self-directed learning as a phased approach or series of events.

Workshop Handout: Designing evidence based capacity building interventions that work for public health The Ontario Public Health Convention, 2019 Page 5

Communities of practice: A single paper reported outcomes for a community of practice which was

focussed on capacity building. The intervention focussed on relationships between researchers and

practitioners (occupational therapists), active learning strategies (reading, face to face, and online

discussion), shared learning and problem solving, with sharing of resources and new strategies. The

study reported knowledge gains as measured pre/post-test. Self-reported changes in practice and policy

were also reported.

Multi-strategy interventions: For the purpose of this review, multi-strategy interventions are those

which explicitly described the intervention as consisting of multiple strategies. The two papers included

in this review defined their interventions as having a training component as well as written materials.

Other components included technical assistance, communities of practice, mentoring/coaching,

internship, appreciative inquiry, meetings, and use of technology. Knowledge and skills were enhanced

by multi-component interventions. One study (rated weak in quality but a useful example) evaluated a

system-level, two year, multi-component intervention aimed at generating system-level change in

evidence-informed trauma care. Post/post evaluation found significant improvement in perceived

readiness and capacity between years one and three.

Implications for practice Overall, the two systematic reviews support the effectiveness of capacity building interventions to

increase knowledge, skills, self-efficacy, changes in practice/policy, application and perceptions of

system-level capacity. However this evidence exists mainly at the individual level. There are a number of

implications from the literature to consider when designing capacity building interventions:

Consider an underlying theory/model/framework when designing your capacity building

intervention. Your organization may currently subscribe to one (or more than one). If not,

consider exploring those that are most relevant to your work. Five are recommended for public

health capacity building as a part of this review. Additionally, explicitly reference the theory,

model or framework and explain how it is being used.

Select the capacity building intervention based on the outcome you are seeking. Incorporate

this into your program planning and evaluation processes by clearly identifying what you are

trying to accomplish as a result of your intervention and then select the intervention type

accordingly.

Optimize delivery of interventions. Consider how you could optimize the strategies you already

have in place, and think about what has been shown to work when planning new strategies. The

literature did find several strategies that can optimize the effectiveness of specific interventions.

For example, with internet based instruction, include discussion groups if your intention is to

improve knowledge, and include practice sessions if your intention is to improve skills. For self-

directed learning, involve participants in selecting learning resources. Also consider using

multiple strategies and interventions.

Evaluate, evaluate, evaluate. There is a need to strengthen the evaluation of capacity-building

interventions in the published literature. More consistent and better evaluations are needed.

Workshop Handout: Designing evidence based capacity building interventions that work for public health The Ontario Public Health Convention, 2019 Page 6

Move beyond pre/post-test approaches and consider methods such as multi-time point data

collection and longer-term data collection. Consider data collection methods beyond self-report.

In addition, the evaluation strategy should:

o Reflect the theory/framework underpinning the intervention;

o Include objective measures of effect;

o Assess baseline conditions; and

o Evaluate features most critical to the success of the intervention.

System level action: More than 10 years after the WHO definition called for capacity building at multiple

levels,1 with one exception, the studies reviewed aimed only at the individual level. Consider, when

planning your capacity building interventions, how you can address the systems in which individuals

operate- the Ecological Framework and the Interactive Systems Framework both provide a structure for

doing so.

Resources Bloom’s Taxonomy Action Verbs. This handy list of verbs, divided into the 6 hierarchies of Bloom’s

Taxonomy, is very helpful for writing objectives for capacity building interventions. Available from:

apu.edu/live_data/files/333/blooms_taxonomy_action_verbs.pdf

The Health Promotion Foundations Course, developed by PHO, contains a useful overview of why and

how health promotion theories, models and frameworks can be used. Available from:

https://www.publichealthontario.ca/en/LearningAndDevelopment/OnlineLearning/HealthPromotion/Pa

ges/HP-Foundations.aspx

We heart health literacy has a useful blog that summarizes many topics relevant to health promotion in

simple language. Their posts include: Diffusion of Innovations Theory

https://medium.com/@whhl/useful-theory-diffusion-of-innovations-a2bad2bac4c8 and Self-Efficacy

https://medium.com/wehearthealthliteracy/i-think-i-can-i-think-i-can-a-tribute-to-self-efficacy-

9dec19da90ff.

These knowledge to action tools are a useful way to determin if and how to apply evidence in your

work:

1. Buffet C, Ciliska D, Thomas H. Can I use this evidence in my program decision? Assessing

applicability and transferability of evidence [Internet]. Hamilton, ON: National Collaborating

Centre for Methods and Tools; 2007 [cited 2019 Mar 4]. Available from:

https://www.nccmt.ca/uploads/media/media/0001/01/110008a2754f35048bb7e8ff446117133

b81ab13.pdf

2. Buffet C, Ciliska D, Thomas H. Will it work here? Tool for assessing applicability and

transferability of evidence (version A: when considering starting a new program) [Internet].

Hamilton, ON: National Collaborating Centre for Methods and Tools; 2011 [cited 2019 Mar 4].

Workshop Handout: Designing evidence based capacity building interventions that work for public health The Ontario Public Health Convention, 2019 Page 7

Available

from: https://www.nccmt.ca/uploads/media/media/0001/01/aaa6890e618d12b9983e156ad89

a582af89c34e6.pdf

3. Moberg J, Oxman AD, Rosenbaum S, Schünemann HJ, Guyatt G, Flottorp S, et al. The GRADE

Evidence to Decision (EtD) framework for health system and public health decisions. Health Res

Policy Syst. 2018;16(1):45. Available from: https://health-policy-

systems.biomedcentral.com/articles/10.1186/s12961-018-0320-2

References

1. Smith BJ, Tang KC, Nutbeam D. WHO Health promotion glossary: new terms. Health Promotion International. 2006;21(4):340-5.

2. Bergeron K, Abdi S, DeCorby K, Mensah G, Rempel B, Manson H. Theories, models and frameworks used in capacity buildling interventions relevant to public health: a systematic review. BMC Public Health. 2017;17(914)

3. DeCorby-Watson K, Mensah G, Bergeron K, Abdi S, Rempel B, Manson H. Effectiveness of capacity buildling interventions relevant to public health practice: a systematic review. BMC Public Health. 2018;18(684)

4. Theory at a glance: a guide for health promotion practiceNational Cancer Institute; 2005.

5. Nilsen P. Making sense of implementation theories, models and frameworks. Implement Sci. 2015;10:53,015-0242-0.

6. McLeroy K.R, Steckler A, Bibeau D. The social ecology of health promotion interventions. Health Educ Q. 1988;15(4):351.

7. Risly K CH. Professional coaching: an innovative and promising leadership development and career enhancement approach for public health professionals. Health Promot Pract. 2011;12(4):497.

8. National Center for Injury Prevention and Control. Understanding the Interactive Systems Framework for Dissemination and ImplementationCentres for Disease Control and Prevention. Available from: https://www.cdc.gov/violenceprevention/pdf/asap_isf-a.pdf

9. Bloom's taxonomy [Internet].: Centre for Teaching Excellence, University of Waterloo; cited 2018 June 12]. Available from: https://uwaterloo.ca/centre-for-teaching-excellence/teaching-resources/teaching-tips/planning-courses-and-assignments/course-design/blooms-taxonomy

10. Bloom's Taxonomy [Internet].: Vanderbilt University Center for Teaching; cited 2018 June 12]. Available from: https://cft.vanderbilt.edu/guides-sub-pages/blooms-taxonomy/

11. Wandersman A, Duffy J, Flaspohler P, Noonan R, Lubell K, Stillman L, et al. Bridging the Gap Between Prevention Research and Practice: The Interactive Systems Framework for Dissemination and Implementation. Am J Community Psychol. 2008;41(3-4):171-81.

Workshop Handout: Designing evidence based capacity building interventions that work for public health The Ontario Public Health Convention, 2019 Page 8