effectiveness and sustainability of social & financial ... · effectiveness and sustainability...
TRANSCRIPT
Effectiveness and Sustainability of Social & Financial Education for Youth
Authors: Daniel Shephard1, Sudha Vijay2 & Aukje te Kaat3
5/1/2017
1 Research Consultant and President at IS-CS Group 2 Research Consultant 3 Research Manager, Aflatoun International
Table of Contents Executive Summary ........................................................................................................................................ i
Methods ..................................................................................................................................................... i
Results ........................................................................................................................................................ i
Implications for Practice ............................................................................................................................ iii
Aflatoun International ........................................................................................................................... iii
Partner Organisations ............................................................................................................................ iii
Other Organizations .............................................................................................................................. iv
Implications for Research .......................................................................................................................... iv
Summary .................................................................................................................................................... v
1. Introduction ........................................................................................................................................... 1
2. Implementation Model ......................................................................................................................... 2
3. Research Methodology ......................................................................................................................... 3
A. Effectiveness Global: Meta-Analysis ................................................................................................. 3
Study Inclusion ...................................................................................................................................... 4
Outcome Extraction ............................................................................................................................... 4
Primary Analysis .................................................................................................................................... 4
Secondary Analysis ................................................................................................................................ 5
B. Effectiveness Case Studies: Qualitative Thematic Analysis ............................................................... 5
C. Sustainability Global: Fuzzy-set Qualitative Comparative Analysis ................................................... 5
Variables and Data Sources ................................................................................................................... 5
D. Sustainability Case Studies: Process Tracing ..................................................................................... 6
4. Effectiveness Results ............................................................................................................................. 6
Meta-analysis and Qualitative findings ..................................................................................................... 6
Positive Impacts ..................................................................................................................................... 7
Positive Trends .................................................................................................................................... 10
Other Results (Meta-analysis and Qualitative Analysis) ..................................................................... 12
Contextual Differences: Meta-analysis Moderation Analysis ................................................................. 17
Meta-analysis Discussion......................................................................................................................... 17
5. Sustainability Results ........................................................................................................................... 19
fsQCA Results .......................................................................................................................................... 19
3
Other qualitative findings related to sustainability................................................................................. 20
6. Conclusions and Implications .............................................................................................................. 26
Implications for Practice .......................................................................................................................... 29
Aflatoun International ......................................................................................................................... 29
Partner Organisations .......................................................................................................................... 29
Other Organizations ............................................................................................................................ 30
Implications for Research ........................................................................................................................ 30
Summary Conclusions ............................................................................................................................. 31
7. References ........................................................................................................................................... 31
8. Annex ................................................................................................................................................... 34
Meta-Analysis Forest Plots ...................................................................................................................... 35
Positive Impacts ................................................................................................................................... 35
Positive Trends .................................................................................................................................... 37
Qualitative and Sustainability studies ................................................................................................. 38
Figures
Figure 1: Countries reporting live Aflateen programmes in 2015 in BLUE ................................................... 3
Figure 2: Future Orientation Forest Plot (Random Effects w/ REML) ......................................................... 35
Figure 3: Resource Management Behaviour Forest Plot (Random Effects w/ REML) ................................ 35
Figure 4: Entrepreneurial Attitudes Forest Plot (Random Effects w/ REML) .............................................. 36
Figure 5: Entrepreneurial Behaviour Forest Plot (Random Effects w/ REML) ............................................ 36
Figure 6: Gender Attitudes Forest Plot (Random Effects w/ REML) ........................................................... 37
Figure 7: Rights Orientation Forest Plot (Random Effects w/ REML) .......................................................... 37
Tables
Table 1: Meta-analyses Results: All Outcomes ........................................................................................... 18
Table 2: Meta-Analysis Study Descriptions ................................................................................................. 34
Table 3: Qualitative and Sustainability Studies Used for Analysis .............................................................. 38
i
Executive Summary The young people of today present unique opportunities and will confront unique challenges. To equip
young people to take hold of these opportunities and meet these challenges, researchers, practitioners,
and policy makers have highlighted the positive impact that life-skills education and financial education
can have on children and young people. International bodies have stated the need for education in such
skills, and a growing number of countries have strengthened the teaching of both social skills and
financial skills in their curricula.
Aflateen is an example of a programme that combines training in social and financial skills. It is a youth
education programme that has been developed jointly by Aflatoun International and its international
network of partners, and in collaboration with The MasterCard Foundation. The programme was
developed in 2011 and has since been piloted, revised, and scaled globally. It is now being implemented
in 64 countries with over 700,000 young people participating in person (in 2015) and over 25,000
learning through the online version, Aflateen Digital.
After this initial period of piloting and scaling-up the Aflateen Social and Financial Education programme
for youth, this report takes stock of the lessons learned regarding the programme’s effects and strategies
for its continued sustainability and growth.
The main objectives of this report are to share those lessons learned and to make recommendations to
Aflatoun International, implementing organisations, other practitioners, and researchers.
Methods
The report synthesizes and summarizes past research on the two broad topics of effectiveness and
sustainability using both quantitative and qualitative methods. The report’s findings are based on 28
studies in 20 countries conducted since 2011 with responses from 5,836 young people, 184 teachers,
and 67 other stakeholders such as staff and school administrators.
The section on programme effectiveness conducts a quantitative meta-analysis and a meta-synthesis of
qualitative study findings to highlight the effect of participating in Aflateen. The meta-analysis
investigated 16 outcomes – both social and financial – while the qualitative section coded the same 16
outcomes as well as several other emergent themes. The sustainability section summarizes a recently
completed fuzzy-set Qualitative Comparative Analysis (fsQCA) by Johannes Meuer and colleagues who
conducted a global investigation of the sufficient and necessary conditions for continuing and expanding
Aflateen programmes. This global study summary is followed by another synthesis of individual
qualitative case studies exploring the factors associated with the successful implementation,
continuation, and expanding of Aflateen.
Results
The effectiveness studies found strong evidence in both the meta-analysis and qualitative data synthesis
on young peoples’ future orientation, resource management behaviour (especially savings),
entrepreneurship attitudes, and entrepreneurship behaviours (p < 0.05). The meta-analysis also found
ii
evidence for positive trends on improving the rights orientation of young participants as well as their
gender attitudes, although the statistical significance of these findings provides less certainty (0.05 < p <
0.10). The qualitative data found even stronger support for Aflateen’s effect on young people’s rights
orientation and gender attitudes, providing additional support for the quantitative trends. Additional
positive effects were found by the qualitative data synthesis, including positive impacts on young
participants’ self-confidence (especially among young women) and financial knowledge (especially
knowledge about loans and budgeting). The qualitative findings also show several drivers behind young
people’s interest in participating in the programme (such as the influence of peers, a desire to make their
communities better, and learn more about their rights, as well as to stand up for others).
There is also room for improvement. The qualitative findings reveal that some students feel that the
financial component of the curriculum is less applicable when they live in poverty. Young people also
struggle to implement successful businesses without additional support. These limitations highlight an
opportunity to align curriculum refinements with each programme area’s economic status to enhance
effectiveness. Additional feedback for improving the programme included the desire of teachers and
facilitators to have more time to learn the programme, more learning aids, and to more carefully refine
to local contexts (beyond the country level).
The sustainability analysis of the first four to five years of the programme are positive with over 70% of
partner organisations who started implementing Aflateen continuing to do so. The fsQCA analysis
identified three strategies for sustaining Aflateen programmes, all of which indicate the importance of
programme fidelity for continuation. The pathways also suggest that funding and high levels of
Secretariat support can function as substitutes for each other in the pathways to sustainability among
new Aflateen partners. This suggests two different funding models, one in which partner funding is the
focus and another in which funding for increased support functions from the Secretariat and other
partners is the focus. For partners with experience implementing Aflatoun’s primary school curriculum, it
is still important that the Secretariat continue to provide support and monitor fidelity – as this was the
only pathway to sustainability for partners who had previous experience with Aflatoun.
The growth of Aflateen has been the result of recruitment of new partners, encouraging current Aflatoun
partners to implement Aflateen, and the expansion of existing programmes. The programme’s midterm
evaluation found that one of the most cited reasons for organisations to start implementing Aflateen
was its flexibility and ability to be merged with their existing programmes as well as its methodology and
content. The fsQCA analysis identified four strategies to grow the size of Aflateen programmes; however,
only two of the four strategies were used to expand programmes that averaged more than 1,000 youth.
As was the case with continuation, these two strategies again highlighted a “high-touch” support (high
service intensity and frequent Secretariat contact) strategy and a “low-touch” funding strategy. Also in
line with the continuation findings, both of the strategies highlighted the importance of high fidelity as it
was a core condition of the high support strategy and a peripheral condition of the funding strategy. The
other two strategies seem suboptimal. One requires low fidelity, which is not desirable from a quality
control stand-point. While the other requires high levels of support, uncertain fidelity, and seems to only
assist in growing small-scale programmes.
iii
At the core of the model for implementing Aflateen is the partnership between the international
Secretariat, international and local civil society, educational institutions, and financial institutions. At
times when funding may be scarce, partnerships have enabled Aflateen programmes to continue. Civil
society organisations with previous experience implementing Aflatoun (CSFE for primary school age-
groups) are particularly resilient in continuing Aflateen with strong technical support from the
Secretariat despite lower levels of funding, as highlighted in the fsQCA pathway on continuation for
previous Aflatoun partners. In addition, civil society partnerships with education institutions can achieve
additional sustainability. National teachers’ organisations can also provide the impetus for national
integration to enable greater sustainability and scale. Furthermore, linking Aflateen partners and
programmes with Central Banks and other financial institution efforts to promote financial inclusion and
education has created additional financial and political support which has enabled the continuation of
the programmes. Partnerships with programmes and policies linked to entrepreneurship also show
promise for supporting Aflateen programmes. This interest can be linked with the findings of our meta-
analysis showing Aflateen’s positive impacts on entrepreneurial attitudes and behaviour.
Implications for Practice
Aflatoun International
Based on the findings above some recommendations for The Aflatoun International Secretariat follow:
a. Create advocacy materials for partners to use that highlight Aflateen’s impact with a
special focus on entrepreneurship.
b. Continue to place focus on its impact on behaviour change and shifting in young people’s
world view and less on traditional knowledge-based financial literacy outcomes.
c. Conduct a cost-benefit analysis to determine if increasing technical support capacity is
more cost effective than direct fundraising.
d. Incentivize higher fidelity programmes by sharing its importance to success, requiring
yearly submission of fidelity information, and publishing fidelity information.
e. Prioritize technical support of Aflateen programmes by partners with experience
implementing the primary school Aflatoun programme.
f. Enhance connectivity between partners by promoting and allocating resources to create
Aflateen specific discussions.
Partner Organisations
We recommend current and prospective Aflateen partners consider the following recommendations as
appropriate for their context:
a. Identify opportunities to link Aflateen programmes to entrepreneurship initiatives,
funding, and policies in their local context.
b. Use the evidence of Aflateen’s impact on future orientation and savings behaviour to
build partnerships with Central Banks and financial institutions.
c. Strengthen partnerships with teachers and teachers’ unions to sustain and expand the
programme. Providing training in the active learning methods of Aflateen can assist here.
iv
d. Conduct needs assessments to understand the needs and capacity of the target groups,
to avoid challenges with content that is too complicated, or not challenging enough, and
to contextualize content.
e. Work to ensure the most vulnerable children are reached, also in rural and remote
areas.
f. Work to improve and track the fidelity of your Aflateen programme because
programmes with higher fidelity have more impact, sustainability, and growth.
g. Make use of the Aflatoun International Partner Portal to share best practices with other
Aflateen partners and to ask questions.
h. If partners graduate programmes (i.e. Aflateen after Aflatoun) they might need more
complex curricula in order to allow for greater gains in learning.
Other Organizations
a. Relevant multinational organisations and national ministries should consider how
Aflateen can support their strategies in meeting the Sustainable Development Goals.
Specifically how Aflateen contributes to
a. Goal 4 targets on skills for entrepreneurship and education on human rights and
gender equity;
b. Goal 5 targets on empowering women economically and eliminating
discrimination against women;
c. Goal 8 targets on reducing the number of youth not in employment, education
or training and increasing the protection of labour rights;4
d. And Goal 17 on enhancing North-South, South-South and triangular cooperation
and capacity development.
b. Teacher training unions and organisations should consider how Aflatoun and Aflateen
partners can provide training on active learning methods for teachers that are applicable
throughout their curriculum.
c. Funders should focus their support on the technical service provision and capacity
building roles of the Secretariat and experienced partners while using implementation
focused funding to encourage new organisations to start implementing Aflateen.
Implications for Research
Aflatoun International, Aflateen partner organizations, and external researchers should focus on both
confirming and expanding the current Aflateen evidence base in the following ways:
a. Impact evaluations of Aflateen should measure future orientation, resource management, and entrepreneurship – especially when conducted in new settings.
b. Future evaluations should also measure gender attitudes and rights orientation to add additional data to confirm the positive trends highlighted herein.
4 The Aflateen curriculum contains learning modules on labour rights.
v
c. Future studies should use the reported effect sizes to determine sufficient sample sizes prior to conducting evaluations. Many of the current studies of Aflateen suffered from small sample sizes that may have been unable to detect important effects.
d. An effort should be made to compile a database of all Aflateen Surveys in order to conduct individual level interaction analysis using demographics and outcomes.
e. Future studies should investigate the interaction between future orientation and resource management behaviour (especially savings) as well as between entrepreneurship attitudes and behaviours.
f. Conduct longitudinal evaluations to determine if outcomes are sustained and to examine how and to what extent Aflateen sustainably impacts quality of life or quality of education.
Summary
Aflateen has expanded throughout the globe touching the lives of millions of young people directly and
indirectly. The combined analysis of evaluations show a strong impact on future orientation, resource
management behaviour, and both entrepreneurial attitudes and behaviours. The programme also shows
positive trends regarding its impact on gender attitudes and rights orientation, although more data is
needed. These positive findings are further enhanced through the outcomes observed in the qualitative
analysis across case studies. This shows the impact of Aflateen to be quite robust. The network should
continue to build on this success by improving the outlook and entrepreneurial spirt of young people
while also focusing on key outcomes that are unique to their country and context.
The research summarized here has identified pathways to continue and expand Aflateen under scenarios
of high and low funding thanks to the constant learning and innovation of the partner network. The
importance of programme fidelity, technical support and capacity building, and partnership are clear
throughout the findings. As a result of its research and partner feedback, Aflatoun International has
already begun restructuring its way of working to better enhance and support these features as part of
its 2016-2020 Strategy. Armed with a breadth of both evidence and experience, it is our hope that the
Aflateen network continues, refines, and expands its strong positive influence on young people.
1
1. Introduction The young people of today present unique opportunities and will confront unique challenges.
Throughout the world, young people make up the single largest segment of the population, represent
the most highly educated generation to date,[1; 2] are innovating at a rate not seen before,[3] and have
developed an expanded sense of tolerance and human rights.[4] This represents a unique opportunity for
countries to leverage the energy, innovation, and equity of a young population to drive progress locally
and globally. However, young people also face many challenges. Globally, secondary school completion
rates are lagging behind primary completion despite the expansion of the knowledge-based global
economy.[2] Young people are also more likely to have to move away from family and friends to find
work, and their work is more likely to be unstable.[5] Young people are unlikely to be saving for their
future[6] and they are more likely to be unemployed than adults.[5] Armed conflict also disproportionately
affects young people due to their bearing the burden of fighting and their loss of education
opportunities.
To equip young people to meet these challenges and take hold of these opportunities, several authors
have highlighted the positive impact that life-skills education and financial education can have on
children and young people.[7-12] In recent years, Ministries of Education and civil society organisations
have sought to bridge the gap between the education delivered to young people and the life-skills and
financial skills they need to address the challenges of their environments.[13-15]
The Aflateen programme seeks to equip young people through five curriculum modules. The modules
focus on (1) enhancing their resilience through self-awareness and an improved confidence in taking
action in the face of obstacles; (2) building a sense of civic responsibility and respect for the rights of
others regardless of gender or cultural background; (3) encouraging sustainable management of financial
and non-financial resources – especially through building savings habits; (4) shifting their orientation to
the future and helping them plan for it; and (5) gaining experience in developing and managing
entrepreneurial projects with social and financial goals. Within the Aflateen programme these five
curricular components are known as (1) personal understanding and exploration, (2) rights and
responsibilities, (3) savings and spending, (4) planning and budgeting, and (5) social and financial
enterprise, respectively.
Aflateen is Aflatoun International’s social and financial education programme for youth and addresses
the above mentioned topics. The programme seeks to socially and economically empower youth to lead
responsible lives and become agents of change. It joins Aflatoun International’s other programmes for
toddlers (Aflatot), for primary-school-aged children in formal education (Aflatoun), and in non-formal
education (Aflatoun NFE). The Aflateen programme is delivered by a global network of Implementing
Partners. Aflatoun’s Secretariat supports its Implementing Partners by providing content, technical
assistance and services, and networking.
In 2011, Aflatoun International and The MasterCard Foundation initiated a 3.5 million dollar partnership
to pilot, roll out, and evaluate the Aflateen programme. After five years of implementation, this report
examines two main aspects of Aflateen:
2
a. When looking globally, what has been the effect of Aflateen on young people’s social
and financial outcomes;
b. What are the lessons learned throughout the network in achieving the continued
sustainability and growth of Aflateen.
The main objective of this report is to share lessons learned and to make recommendations to Aflatoun
International, implementing organisations, other practitioners, and researchers.
2. Implementation Model The Aflateen programme is implemented through Aflatoun International’s bottom-up network of
organisations interested in education. Local organisations, international organisations, teacher training
institutes, and ministries are members of the network. The Aflatoun International Secretariat provides
technical support and tools to partner organisations to assist them in delivering life skills and financial
education to children and youth. The Secretariat arranges regional and international knowledge sharing
opportunities for partners each year to help disseminate best practices.5 It also coordinates and supports
the Regional Master Trainers around the world to provide high quality training. The current Aflateen
training manual was based on the Aflatoun manual (primary school) and adapted with additional
facilitation skills for youth.6 Additional technical support services provided by the Secretariat to partners
include tools and direct support on implementation; monitoring, evaluation, and research; and
fundraising. Due to the organizational priority of integrating into national programmes and scaling
current programmes, Aflatoun International assists organisations with their advocacy efforts on national
and regional levels. Where suitable and possible the organisation also helps implementing organisations
in the network with finding funding opportunities, proposal writing and the creation of national or
regional partnerships to achieve their goals.
One of the unique aspects of the Aflateen programme is its flexibility which allows partners to merge the
programme into other interventions. This flexibility has been noted as a key strength of the programme
and has assisted the expansion of Aflateen and resulted in unique versions of Aflateen reaching unique
populations.[16]
The Aflateen programme initially aimed to reach 250,000 youth in 50 countries in 2016; it reached
775,198 youth (64% female) in 64 countries in the year 2015, surpassing the initial objectives. These
young people were reached through 85 reporting partners.[17] Aflateen has been implemented with
young people affected by conflict, for example in Syria and the Democratic Republic of Congo. It has
been integrated in projects focused on empowering young women, for example in Tajikistan with Mercy
Corps[18] and in partnership with Plan International UK and Credit Suisse in Brazil, Rwanda, India, and
China. Aflateen has also been combined with rural entrepreneurship projects seeking to empower young
5 Regional and international meetings alternate on a biannual basis. 6 This adaptation was undertaken during the midterm evaluation of Aflateen at the suggestion of partners.
3
people to explore innovative agricultural professions, for example in Uganda7. Finally, it has been
integrated into national curricula, for example in Peru and Cameroon.
Figure 1: Countries reporting live Aflateen programmes in 2015 in BLUE
3. Research Methodology In order to examine the effectiveness and sustainability of the Aflateen programme, several
methodologies have been used, including quantitative and qualitative measures. To address the
effectiveness of Aflateen the methodology of the global meta-analysis and qualitative case studies are
used and described below. To address the topic of sustainability the methodology used for a global
analysis fuzzy-set Qualitative Comparative Analysis, fsQCA is described followed by the process tracing
methodology that looks at the national integration process of Aflateen.
The Aflateen research referenced in this report covers 28 studies in 20 countries conducted since 2011
with responses from 5,836 young people, 184 teachers, and 67 other stakeholders such as staff and
school administrators.
A. Effectiveness Global: Meta-Analysis
The meta-analysis sought to address the overall impact of Aflateen as measured around the globe by
studies completed and analyzed prior to fall 2016 on 16 key outcomes of interest. The outcomes covered
were: (1) Civic Attitudes, (2) Civic Engagement, (3) Educational Aspiration, (4) Entrepreneurial Attitudes,
7 A needs assessment was conducted in 2015 to understand the needs of the target group to better align the modified financial literacy curriculum for the agricultural sector, and to inform future project design. See more here.
4
(5) Entrepreneurial Behaviour, (6) Financial Knowledge, (7) Future Orientation, (8) Gender Attitudes, (9)
Health Knowledge, (10) Pro-Social Attitudes, (11) Pro-Social Behaviour, (12) Research Management
Attitudes, (13) Resource Management Behaviour, (14) Rights Orientation, (15) Rights Oriented
Behaviour, (16) Self-Efficacy / Confidence.
Study Inclusion
This meta-analysis included studies conducted by Aflatoun and its partners on the effectiveness of
Aflateen. Studies had to either measure changes in the outcomes of interest over time or conduct
comparisons with a counterfactual. The types of studies include one randomized controlled trial (RCT)8
and seven studies comparing outcomes before and after a period of participation in Aflateen.9 These
eight studies covered seven countries with two studies from Saudi Arabia.
In total, the eight studies included 2,446 participants at the baseline data collection with varying levels of
responses at the follow-up data collection depending on the outcome.
Outcome Extraction
Data were extracted from the study reports and the raw datasets by the first and second author.10
Specific questions from each of the studies were matched to the above listed key outcomes by the first
author with confirmation by the Aflatoun research manager in two iterations.11
The driving criteria for assigning variables from individual studies to outcomes in the meta-analysis were
(1) conceptual similarity between the variable and the meta-analytic outcome and (2) consistency across
studies such that the same question/construct was extracted from all studies that shared that
question/construct in common. A maximum of one question/construct was extracted per study, per
meta-analysis outcome. This was done to avoid inaccurate degrees of freedom and confounding due to
covariance. Outcomes were included in the meta-analysis without regard to their level of statistical
significance or the direction of impact.
Primary Analysis
The primary research question of Aflateen's impact on the 16 outcome areas was assessed using
separate meta-analyses of the included studies. All studies that could be included were included. For
many outcomes; however, fewer than eight studies measured the outcome of interest as can be seen in
the results.
8 A second RCT was conducted in Peru to compare two different methodologies of Aflateen. Since it was designed to measure the impact of adding a planning module it is not included in this meta-analysis which is concerned with the effectiveness of Aflateen as a whole. Interested readers are directed to read the forthcoming study by Matthew Bird: SMART Goals, Gender, and Youth Financial Education: Evidence from a Randomized Experiment in Peru. 9 Another study conducted in Indonesia was not included because of differences in data collection methods used between the baseline and endline as well as the short interval between the two data collection points. 10Daniel Shephard and Sudha Vijay. 11 To reduce the chance of unknowingly biasing the results, the research manager did not have visibility on how decisions would impact the meta-analysis findings.
5
Secondary Analysis
For outcomes that were at least marginally significant (p < 0.10) in the meta-analyses, meta-regressions
were run to determine if there were contextual differences that could explain differences between
studies. The four variables used for moderation analysis were based on theory and previous research
and included months of follow-up ("Months"),[19; 20] average age of the participants ("Age"),[21] and
whether the methodology was an RCT or a pre-post study ("Method").[22]
B. Effectiveness Case Studies: Qualitative Thematic Analysis A number of studies have used qualitative research methods to examine outcomes of the Aflateen
programme, or to examine the process and challenges around the implementation. Therefore, carrying
out qualitative analysis to aggregate findings from research interviews allows for a more holistic view
about participants’ subjective thoughts and opinions on the Aflateen programme, globally.
In total, 9 qualitative research reports were analysed from Egypt, Indonesia, Kenya, Nepal, India,
Tajikistan, The Gambia, Democratic Republic of Congo and Georgia.
A qualitative software package, Dedoose, was used to code the reports for a priori (i.e. quantitative
constructs) and emergent codes (i.e. implementation of Aflateen, curriculum challenges and limitations
related to the content, reasons for joining and the corresponding challenges that impacted joining and
future focus/improvements).
C. Sustainability Global: Fuzzy-set Qualitative Comparative Analysis
In 2015-2016 Aflatoun International conducted a study “To Continue and Grow: Final Evaluation of The
MasterCard Foundation Partnership with Aflatoun on Youth Social and Financial Education.” It
investigated the factors that resulted in partner success related to Aflateen programme continuation and
growth.[23] For this, the core of this previously completed study, led by Johannes Meuer and colleagues,
made use of an innovative methodology that combines quantitative and qualitative data analyses: Fuzzy-
set Qualitative Comparative Analysis (fsQCA). This methodology makes use of Boolean algebra and set
theory to identify combinations of factors that are sufficient and necessary for achieving a particular
outcome.[24; 25] The methodology was used to identify multiple pathways and was then followed up with
in-depth case studies to explain cases that align with the identified solutions and any deviant cases.
The use of fuzzy sets enables the researchers to make more nuanced determinations of whether or not a
particular case is a member of a set, or group. Each case is given a score for each outcome and
explanatory factor ranging from 0 to 1. In this range, 0 represents complete exclusion from a given
group, 0 < 𝑥 < 0.5 represents being mostly out of the set, 0.5 represents ambivalence, 0.5 < 𝑥 < 1 the
unit being mostly a member of the group, and 1 represents complete group membership.
Variables and Data Sources
The cases used in the analysis were Aflateen programmes. The sources of data were the annual surveys
conducted by Aflatoun International on programme outreach as well as interviews with regional
programme managers. The two different outcomes analysed were the continuation of an Aflateen
6
programme and the growth of the Aflateen programme. Continuation was coded as 0 for programmes
that ended after one year up to a full membership score of 1 for programmes that are ongoing for at
least three years. Growth was coded based on each case’s place in the distribution of their compound
annual growth rates, and as such it should be interpreted as a measure of relative growth. Membership
scores were distributed based on anchors of 0 for -100% growth; 0.5 for -0.3% growth; and a full
membership score of 1 for at least +25.7% growth. The five potential explanatory variables selected for
analysis were: (1) service intensity representing the given Aflateen partner’s utilization of the
Secretariat’s services; (2) prior experience implementing the primary school Aflatoun curriculum prior to
starting Aflateen; (3) frequency of contact between the partner and the Secretariat; (4) funding
availability to support the implementation, continuation and growth of the partner’s Aflateen
programme; and (5) fidelity of programme implementation.
D. Sustainability Case Studies: Process Tracing Process tracing as a research method has been defined as ‘the systematic examination of diagnostic
evidence selected and analysed in light of research questions and hypotheses posed by the
investigator.’[26] To understand the factors that led to scale up (national integration) of social and
financial education in national education curricula, this method was used in Cameroon and Kenya. The
two studies aimed to understand which conditions facilitated that process, the main lessons learned, the
main challenges, and to what extent and how Aflatoun International and its partners contributed to this
process.
4. Effectiveness Results The following summarizes the impact of Aflateen on programme participants as identified through a
meta-analysis of quantitative evaluations and a synthesis of qualitative case study results. The following
provides a summary of the global, aggregate impact of the Aflateen programmes being implemented and
evaluated by partners around the world.
Meta-analysis and Qualitative findings The combination of results from around the globe show positive impacts on four outcomes based on
both quantitative and qualitative data: Future Orientation, Resource Management Behaviour (i.e.
Savings Behaviour), Entrepreneurial Attitudes, and Entrepreneurial Behaviour. Two other outcomes
show positive trends: Gender Attitudes and Rights Orientation.12 No negative results were found on
aggregate in the quantitative data. On a number of indicators, no effect was found, see “other results.”
The following describes these six results in more detail. The Table 1 below presents the results of all
meta-analyses.
12 Another outcome (Health Knowledge) also showed a positive impact. However, it was only evaluated in Tajikistan as part of a programme that combined Aflateen’s five traditional modules with 2 other modules on Sexual and Reproductive Health. Further details are therefore excluded. However, we encourage interested readers to find more details on this outcome by referencing the forthcoming paper by Karimli and colleagues.
7
The qualitative analysis that follows each outcome builds upon the meta-analysis to give a more robust
picture of the significant results. These findings will be discussed by highlighting quotes from country
reports that best stress the underlying meaning of each code. A later section will discuss additional
findings and themes from the qualitative synthesis.
Positive Impacts
Impact on Future Orientation
Aflateen had a substantial positive impact on participants' attitudes about the future, also known as
future orientation. Participants' attitudes increased by 0.748 standard deviations across 7 studies (95% CI
= [0.039, 1.458]; p-value = 0.039). This is roughly equivalent to moving from the 50th percentile to the
75.4th percentile. For example, the median student at endline in the Aflateen programme would have
higher scores on the attitude measure than 75% of baseline/control group students. See Figure 2 for
details.
Several case studies complement the quantitative findings showing Aflateen to have a positive impact on
participants’ Future Orientation. Aflateen positively motivated participants to determine their
professional interests and explore their desires to become something. One participant in Kenya said:
“…it has taught me how to plan for my future through setting goals. The club has
encouraged me to work hard so as to achieve my goal of becoming the Kenyan Police
Force.”
Another participant in Nepal stated that certain exercises carried out (as part of the Aflateen
programme) helped them become more aware about themselves and what they wanted in their life:
“…it has helped me learn more about my strengths and weaknesses, and helped me to
determine what I really want to do in my life…”
Impact on Resource Management Behaviour
Aflateen had a large positive impact on participants' self-reported resource management behaviour –
namely their savings behaviour. Participants' self-reported behaviour increased by 0.992 standard
deviations across 7 studies (95% CI = [0.162, 1.823]; p-value = 0.019). This is roughly equivalent to
moving from the 50th percentile to the 83.7th percentile. See Figure 3 for details.
Studies similarly complement this positive effect. The programme helped participants grow in their
understanding of planning and budgeting pertaining to saving. It positively affected students’ desire and
ability to save money, where previously they would have spent the money given to them by their parents
frivolously on other unnecessary items. One participant in Tajikistan said:
“Our parents give us some money and we used to spend it without any planning, the
program gave us great idea of planning the budget”.
Another participant in Kenya similarly said:
8
“In the club, I was educated on the importance of saving and my mind broadened. I
began saving and didn’t ask money from my parents all the time. I learned how to
sacrifice the non-important things I used to buy for leisure …”
The programme additionally assisted participants in setting priorities and saving money to use for other
purposes such as helping their parents or pay for school related items (i.e. tuition, textbooks etc.). This
reveals their increased sense of responsibility and ability to set spending priorities. One participant in
Nepal said:
“Few weeks ago, I saved Rs 1500 of selling fruit. My mother got sick and I used that
money for medication.”
The Aflateen programme also impacted participants to save non-monetary items, especially in regions
with economic difficulties. A participant in Nepal said:
“I have learnt to save toothpaste, shampoo, light etc. It may seem like small and simple
things, but it really makes a bigger impact on your way of saving…”
Additionally, an impact of resource management was also evident among girls, especially in Egypt and
Tajikistan13. This highlights a positive impact of the Aflateen curriculum on girls, especially for those
coming from the more reserved side of the social strata. In Egypt it was evident that girls saved more:
“Of those not saving, most are boys, since their overall spending is more than girls.”
The curriculum also made girls cognizant of the importance of saving money, empowering them to save
for their own purposes i.e. to support themselves and their families. One participant in Tajikistan said:
“…. some adolescent girls started planning their activities for the next day and
practicing saving. Some of them already purchased jewelleries, sheep and actively
doing budget plan for themselves and their families.”
The strength of the curriculum is that it allows for girls to get together and influence one another to save
money. Another observation was made by a teacher in Egypt:
“…mostly girls, save through Aflateen’s group saving scheme, reporting that it makes
them encourage each other to save...”
However, despite the above-mentioned positive outcomes some students voiced their frustration with
the relevancy of the curriculum for their life situations. Economic factors were a significant limitation.
One teacher in Nepal stated:
“…there were participants who mentioned that they didn’t have sources of income, and
there was no point in learning to save money.”
13 Please note that there were only girls in the Tajikistan study..
9
Although non-monetary savings were an alternative in the face of economic hardship some students felt
like it did not really help them to support themselves. These students wished for more direct help from
others to start their business and earn money. A participant in Nepal said:
“… saving non-monetary resources sound appealing but it really doesn’t help me
support my living. There are so many organizations working for people living with HIV,
and if they really want to work for us, they could help us to start out own small business
or link up with job opportunities.”
Impact on Entrepreneurial Attitudes
Aflateen had a substantial positive impact on participants' attitudes about their desire and ability to start
an enterprise. Participants' attitudes increased by 0.608 standard deviations across 7 studies (95% CI =
[0.132, 1.084]; p-value = 0.012). This is roughly equivalent to moving from the 50th percentile to the
70.7th percentile. See Figure 4 for details.
Also in Nepal, positive findings were evident in the qualitative analysis with participants being
enthusiastic about starting a business in the future, either with friends or independently. One
participant said:
“I have talked to my other friends, who participated in Aflateen to do more saving and
do collective investment to start a boutique.”
Some participants were motivated by the idea of starting a business as a means to better their lives. One
student in Nepal said:
” I am now interested to learn financial skills and start my own enterprises. As an HIV
positive person, I want to find out better ways for my livelihood.”
Additionally, an impact was also evident amongst girls. For example, as a result of the programme, girls
in Tajikistan, had a desire to set up their own business with friends. This would enable them to help
support their families.
“We (3-4 of us) can open our own bakery program, bake cakes, cookies and sell and
support poor families.”
Impact on Entrepreneurial Behaviour
The impact on entrepreneurship was not confined to attitudes, there was also a smaller positive impact
on the entrepreneurial behaviour of these youth participants. Participants' self-reported entrepreneurial
behaviour – such as having run a small enterprise in the past 12 months – increased by 0.316 standard
deviations across 5 studies (95% CI = [0.185, 0.448]; p-value = 0.000). This is roughly equivalent to
moving from the 50th percentile to the 60.8th percentile. See Figure 5 for details.
The qualitative findings complement this effect with participants setting up their own business to earn
money, make a profit and become self-sufficient. One striking example is from a participant in Nepal who
10
set up a kiosk by consulting with other kiosk owners to research what to buy and what not to. This
showcases initiative as well as the motivation to set up a successful entrepreneurship:
“My kiosk has made me a role model in my village and people are surprised at how I
have so many customers. Before consulting what to buy for my kiosk I would consult
the biggest kiosk around and they would share their ideas.”
Another example is of participants collaborating with others who were not part of the Aflateen
programme to help run their business. For example, in Georgia:
“One of the projects included home delivery service for certain extra cost by Aflateen
participants with help of their friends and neighbours. Participants delivered cakes from
the bakery.”
However, despite participants taking the initiative to start their own business independently some did
require the support of their parents and teachers to “kick-start” it. In Kenya, one set of students were
able to set up their enterprise through the support of their parents and teachers.
“…through support of parents students are able to initiate enterprise: because of the
money saving aspect, parents support us with money we use in clubs. Parents and
teachers buy vegetables from our club farm.”
This highlights the need for a support system in certain circumstances, to enable students to fulfil their
entrepreneurial goals.
Additional challenges faced by those who set up their own business included suffering losses due to
factors not readily controlled such as clients not paying or damage to their business. A student in Kenya
said:
“I would face consequence during the selling of mandazi because some of the students
would take them on credit and not pay back. I had difficulties in finding money to face.
Also some of the vegetables were attacked by pests and the money wasn’t enough to
cater for pesticides. This caused me a loss.”
Positive Trends
While participants' gender and rights attitudes did not reach the traditional cut-off for statistical
significance, the data still present positive trends (0.05 < p > 0.10) when aggregated across all studies.
Gender Attitude
There was a substantial positive trend indicating that participating in Aflateen results in more equitable
gender attitudes among participants (SMD = 0.664; 95% CI = [-0.051, 1.379]; p-value = 0.069) as can be
seen in Figure 6.
Moreover, the qualitative analysis showed a positive effect of Aflateen on Gender attitudes (i.e.
perceptions about girls’ roles and opportunities). Gender relations between girls and boys became
11
better, with perceptions about one another becoming more positive. There was a difference in how girls
were viewed before and after implementation of the programme – with girls being viewed more
positively regardless of their proximity to a boy. A participant in Egypt said:
“…before Aflateen, youth presumed that any girl who stands with a boy to have bad
intentions and assume they are doing something against the conservative values and
traditions of the society. But after Aflateen, they treat each other differently and are
more cooperative and interactive…. there was a change in perceptions of both girls and
boys equally on gender roles and relations”
Another participant in Kenya similarly said:
“The club has taught me….[how to] also deal with boy/girl relationships better and I
have learnt to ignore unnecessary relationships [with unhelpful friends].”
However, not all case studies showed a positive impact of Aflateen on gender attitudes. This was
particularly evident in Georgia wherein traditional cultural views seemed prevalent throughout the
community. Gender stereotypes were discussed with a somewhat negative attitude towards girls,
showing a need for further sensitizing workshops in this domain.
“Boys are more privileged than girls. We cannot have gender equality in everything.
Women are not allowed to do everything men do.”
There were also issues concerning implementation of the gender attitude component as it was
considered “taboo”. It made it difficult to discuss such topics in class, as schools were not ready for it.
One participant in Indonesia said:
“It is difficult to discuss things relating to gender and sexuality as our society restricts
us from openly discussing these topics.”
Rights orientation attitude:
There was also a large positive trend indicated in participants' rights orientation (e.g. believing that
everyone has equal rights). This trend is promising because the estimate’s 95% confidence interval is
almost entirely positive (SMD = 0.796; 95% CI = [-0.004, 1.596]; p-value = 0.051) as can be seen in Figure
7.
A more positive effect was prevalent in the qualitative analyses across case studies. This stronger
qualitative finding may be because of the need to provide an open opportunity to explain how the rights
orientation is applied in context. Students vocalized that the programme made them more cognizant of
their rights and responsibilities, particularly those pertaining to education and their freedom to move
and socialize with others. One participant in Kenya said:
“I have learned about rights and responsibilities such as the right to education, basic
needs, self-expression and freedom to move and socialize with others.”
12
As a result of this newfound mindset, the analyses revealed that students were more confident and
therefore, more inclined to stand up for their rights and rights of others. This goes beyond a conceptual
and theoretical manifestation of a rights orientation attitude. For example, a case study in India, that
used observations as one of the research methods, found that students in schools where the Aflateen
Programme took place, were more vocal about their rights, compared with the control group:
“Students from intervention schools are observed to be much more vocal about gender
equality, and the right to protection against caste or religion…”
Students also used their knowledge about their rights and rights of others to make decisions for
themselves. One participant in Gambia said:
“There was a positive change in rights orientation of young people with qualitative
results revealing young people making decisions based on knowledge of their rights.”
However, Aflateen didn’t always positively impact student’s thoughts and opinion pertaining to their
rights orientation. This might have been exacerbated by the educational context wherein students’
learnt about human rights as part of their schooling but in theoretical terms. Thus, despite Aflateen,
students may struggle to detect any rights violations in real life settings. One teacher in Georgia said that
the students were not ready to understand what human rights violations and inequality are.
“As about practical implications, they are not well equipped with practical knowledge
to detect human rights violation and inequality.”
Other Results (Meta-analysis and Qualitative Analysis)
For the following outcomes, there was not enough quantitative data to determine if there was an effect
across studies in the meta-analysis. This indicates that either the programme has no effect on these
outcomes (as measured), that the effect is too small to detect given the total combined sample size of
the included studies, or that there was too much variance in the effects – with positive effects in some
contexts and negative or null effects in others. The aggregate data did not support any global effects of
the programme on the outcomes of: Pro-Social Behaviour, Self-Efficacy/Confidence, Financial
Knowledge, Resource Management Attitudes, Civic Attitudes, Rights Oriented Behaviour,14 Educational
Aspiration, Civic Engagement, and Pro-Social Attitudes. It should be noted that individual programmes
did show effects on some of these outcomes; however, there were no consistent global effects on
aggregate.
On the other hand, the qualitative analysis revealed a positive effect on outcomes not significant in the
meta-analysis. These are discussed below
Pro-social behaviour
Aflateen showed a positive impact on participant’s pro-social behaviour. Some interesting findings found
14 As distinct from their rights oriented attitudes. For example, in Tajikistan the outcome measuring the freedom of movement of participating young women was used.
13
that students’ behaviour with their parents, peers and teachers improved as a result of Aflateen.
Student’s, who previously misbehaved and disrupted others, became more respectful and cognizant of
their studies. One participant in Kenya said:
“…before I joined Aflateen club I would get suspended from school because after the
deputy issues punishment I would challenge her. At home, I would misbehave…but this
has changed…. my grades have improved and I know how to appreciate things.”
A case study from Egypt similarly revealed that students who were troublemakers at home and school,
became respectful of others, especially those with disabilities. This helped them develop better
friendship skills.
“My friends and I used to make fun of our schoolmates who have disabilities, but now
we help them to become stronger and better people and defend them from bullying by
other youth.”
This behaviour extends to respecting other religions and cultures. One participant in Nepal said:
“The trainer told us about respecting other religion and cultures….after the training, it
was clear to me about importance of respecting each other’s religions and cultures”
However, not all case studies showed a positive impact of Aflateen on pro-social behaviour due to the
persistence of cultural influences. Some students in Georgia stated that despite Aflateen, religious
differences in the classroom still impacted their ability to interact and communicate with some of their
peers. Sometimes family views were also negative against those from different religions. One participant
said:
“….have negative attitudes towards persons of different religious views.… [parents] are
concerned trying to protect [us] from the influence of persons with different religion.”
Civic behaviour:
Focus group interviews across various case studies showed that the curriculum instilled its participants
with a desire, and confidence, to tackle social problems, to make their community better. One striking
example from Egypt showed Aflateen youth coming together to improve the basic conditions of a local
healthcare unit:
“Aflateen youth took photos of the health unit and wrote a report about it and sent it
to a national newspaper. A week later, reporters from the newspaper came over and
made coverage about the hospital. As a result, members from the community were
motivated and mobilized to clean the health unit, in addition to receiving financial and
in-kind contributions from community members to improve the basic conditions of the
unit.”
A similar example was evident in Kenya with students coming together to clean their environment:
14
“My environment was polluted with litter everywhere. My friends and I have organized
a club to ensure that we have a clean environment.”
This social consciousness extends to students’ sharing their experience of Aflateen with others so as to
pass their learning and understanding on. One participant in Nepal said:
“I understand the importance of the Aflateen programme, and I wanted to share my
experience with my friends…[we] form Unity Aflateen Group. I am in the working
committee and I have organized orientation sessions on Aflateen for friends.”
Rights orientated behaviour
The Aflateen curriculum increased students’ awareness of their rights and gave them the confidence and
the desire to claim them. One participant in Egypt said:
“…my teacher used to enter the class and not explain or smoke in class and we were
afraid to tell him anything. Now, after Aflateen, we became courageous and we tell him
that he needs to explain the lesson and if he does not respond, we gather ourselves and
complain to the Headmaster. Before, we were afraid to do so…”
Aflateen also seemed to play a key role in enhancing not only girls’ knowledge of their rights but also
how to protect themselves, particularly against unwanted sexual advances. One participant in Kenya
said:
“I have also learned about my rights as a youth. If a boy approaches me with sexual
advances, we are able to say no. It is possible to prevent ourselves from such dangers
like rape by not walking alone.”
Confidence
Student’s confidence was positively impacted by the Aflateen curriculum with students becoming more
confident in expressing their thought’s and opinions to adults, including their teachers, and their peers.
One student in Kenya said:
“…also didn’t know how to express myself well but thanks to this club, I have new-found
confidence. I can now communicate well with my fellow students and teachers and can
address a crowd of people confidently.”
A similar example was evident in Georgia:
“…gained communication and self-confidence through Aflateen.”
There was also an observed impact on confidence interacting with individuals of the opposite sex. One
participant in Kenya said:
“We were taught a lot of life-changing things like how to associate with people and…I
also learnt how to deal with boy-girl interactions.”
15
Aflateen also positively impacted girls’ confidence, particularly those coming from more conservative
communities. Girls who were previously shy prior to joining Aflateen, and had low self-esteem, gained
more confidence. One teacher in Egypt observed:
“After participating in Aflateen, many girls became more outspoken, confident, and
interactive with other youth, which noticeably improved their self-esteem. Hence, the
impact of the programme on self-confidence is more evident on girls than for boys...”
Civic attitude-perceptions about role in improving community
Aflateen instilled students with a positive attitude regarding their role in improving community. One
participant in Kenya said:
“We are also planning on opening a canteen which will give us profit and help us
improve our living standards. It is also increasing the prestige of our school.”
Similarly, in India:
“Aflateen …sensitizes students about using money for cooperation and community.”
Students learnt to be active agents of change in society and how to solve the problems in their
community. Particularly in Egypt learning how to design and conduct campaigns to solve community
problems was highly valued by youth and they determined to conduct cleaning campaigns during the
summer holidays to solve the solid waste problem that prevails across the four villages. This also allowed
them to understand their communities better, and its needs, which they never thought about previously.
Participants in Egypt said:
“…we now feel that we are active agents of change in our communities and we have a
responsibility to solve our village problems”.
Another participant from Egypt similarly said:
“…originally, I wanted to become an interior designer when I finish my education. After
Aflateen, I decided that I want to do something more beneficial for my community."
Resource management attitude
Qualitative analyses showed a positive effect of Aflateen on resource management attitudes. Students
felt they understood the importance of saving money. They developed an appreciation of the value of
money and subsequently became cognizant of spending on unnecessary items. This generated a positive
savings attitude on other resources too. One participant in India said:
“… not only makes students aware of the importance of saving various resources like
water, food, electricity, but it helps them to translate this into practice.”
This newfound positive attitude towards saving money also made students more responsible towards
securing their future. One student from Egypt said:
16
“If my parents cannot afford to pay for my education, I have to think about alternatives
like saving or working over the summer to get my education fees”.
Some participants wanted to channel their positive attitude for resource management to assist others. A
striking example was a case study in Nepal where HIV infected youth participated in Aflateen were
motivated through their understanding of financial education to assist others with similar health
concerns.
“I have understood the importance of social and financial education for youth infected
and affected by HIV. I really want more of them benefitting from this programme.”
A positive impact on gender was also evident with girls mentioning that Aflateen made them more
aware about saving (including where to save) and budgeting. One participant in Tajikistan said:
“Every person should have some savings and I think that bank or school safe is the safest
place to keep money.”
Financial Knowledge
In the case studies that did touch upon financial knowledge relating to planning, budgeting, loan
schemes, and calculating profit, students voiced a newfound sensitivity for such knowledge. One
participant in Kenya said:
“As an individual and also to the group the education has helped gain the knowledge
of calculating the profit on the loaned amount, thus getting some knowledge and skills
that are provided by the bank.”
Similarly, in Congo, one facilitator said:
“Students developed an ability to create and implement a budget for their finances….”
A positive effect of Aflateen on Financial Knowledge was not prevalent in all studies. This was because it
was not adequately covered as part of the curriculum in certain contexts. Teachers in Egypt mentioned a
need to revise the curriculum to potentially better incorporate it as well as to make it more relevant to
the context.
“…[need to] revisit the financial component in the Aflateen curriculum and modify it to
make it more relevant to the context in which the programme operates. This will
undoubtedly increase efficiency by enabling youth to apply all what they are being
taught.”
Educational aspirations
Students who benefited from Aflateen also showcased positive educational aspirations relating to their
future. This was evident in Congo:
“Students who took part in Aflateen developed a desire to pursue higher studies in
comparison to those who did not partake in the programme.”
17
This was similarly true in Georgia:
“Participants goals…included higher education…”
One student in Kenya realized about the importance of studying to achieve her dream to become a
doctor. This resulted in her reorganizing her studying schedule to ensure she completed assignments in
time.
“I also came to realize that studying is not done in school alone. My goal to become a
doctor is helping me achieve it. I came up with a schedule to help me finish assignments
in time…”
Contextual Differences: Meta-analysis Moderation Analysis Generally, the moderation variables did not explain differences among studies. The only exception was
that the months between baseline and follow-up data collection only explained the heterogeneity in the
entrepreneurial behaviour outcome. Longer follow-ups seem to provide participants with more time to
put their entrepreneurial ideas into practice; however, the effect is so small it has no substantive impact
(SMD = 0.008; 95% CI = [0.001,0.014]; p-value = 0.017). This small finding may just be an artefact of the
structure of the Aflateen intervention in which the Social and Financial Enterprises module is last, or an
artefact of running 22 moderator tests.
Despite this small moderation effect on one outcome, the more important finding is that the moderators
did not explain the heterogeneity in outcomes in any case other than the one noted. This may be
because there are other country-specific variables that explain the differences between studies and
could not be distinguished because there was only one study per county (except for Saudi Arabia). The
lack of significant moderation effects for follow-up, age, and type of intervention is noteworthy given the
literature to support the impact of these variables on this study’s outcomes of interest.
Meta-analysis Discussion
The study suffers from several limitations that may introduce bias into the overall findings. The selection
of outcomes to combine in the meta-analysis based on the criteria above, although systematic, is still a
matter of judgement that may introduce bias. This is particularly true in our case because three of the
studies included herein used different measurement instruments (Tajikistan, China, and the first Saudi
Arabia study) than the other five studies. The authors attempted to reduce the chance of bias being
introduced by having the selections of variables to include in each outcome reviewed by an author who
did not conduct the meta-analysis. However, the lack of two independent, blind data extractions may
have introduced bias into the results. Another source of bias is the use of self-report instruments that
had only been partially validated15 and were then used in new contexts and languages.
Despite these limitations, the reported meta-analyses provide findings regarding the Aflateen
programme that are more robust than a single study. They indicate that there are positive effects on
future orientation, resource management behaviour, and entrepreneurship that span across a diverse
15 Some constructs and questions had been validated, others had not.
18
network of partners and countries. These positive impacts did not change when conducting a robustness
check using a Trim & Fill Analysis to determine if missing data points could shift the finding. In all cases,
the positive and significant findings remained.
The positive findings regarding resource management behaviour and entrepreneurship behaviour align
with a previously conducted systematic review and meta-analysis of financial education programmes for
children and youth that found the Aflatoun programme to have one of the largest impacts on financial
behaviour.[27]
The additional positive trends regarding Aflateen’s impact on shifting the ingrained gender attitudes and
rights orientation are also encouraging considering the importance and difficulty in measuring and
shifting these attitudes.
Table 1: Meta-analyses Results: All Outcomes
studies estimate ci.lb ci.ub se zval pval I2
Gender Attitudes 8 0.664 -0.051 1.379 0.365 1.821 0.069 99.5
Pro-Social Behaviour 7 0.611 -0.267 1.489 0.448 1.364 0.172 99.4
Self-Efficacy / Confidence 7 0.455 -0.177 1.087 0.322 1.411 0.158 99.3
Future Orientation 7 0.748 0.039 1.458 0.362 2.068 0.039 99.3
Resource Management Behaviour 7 0.992 0.162 1.823 0.424 2.343 0.019 99.6
Entrepreneurial Attitudes 7 0.608 0.132 1.084 0.243 2.502 0.012 98.2
Financial Knowledge 6 0.284 -0.583 1.151 0.442 0.642 0.521 98.9
Research Management Attitudes 6 0.468 -0.149 1.085 0.315 1.487 0.137 98.7
Rights Orientation 6 0.796 -0.004 1.596 0.408 1.951 0.051 99.5
Entrepreneurial Behaviour 5 0.316 0.185 0.448 0.067 4.718 0.000 52.2
Civic Attitudes 3 0.442 -0.092 0.977 0.273 1.621 0.105 96.9
Rights Oriented Behaviour 2 0.089 -0.022 0.201 0.057 1.570 0.116 0.0
Educational Aspiration 1 -0.047 -0.164 0.071 0.060 -0.781 0.435 0.0
Civic Engagement 1 0.053 -0.063 0.169 0.059 0.894 0.371 0.0
Pro-Social Attitudes 1 -0.066 -0.182 0.050 0.059 -1.113 0.266 0.0
Health Knowledge 1 0.153 0.034 0.271 0.060 2.526 0.012 0.0
These are all the models of the outcomes. If there is one study a Fixed Effects model is used. If there are more than one a Random Effects model is used with REML.
Effectiveness Research Summary
The qualitative findings suggest that Aflateen positively impacts students’ attitudes and behaviours.
There is very little evidence of negative impacts, although there are some non-effects. This highlights an
underlying strength of the Aflateen curriculum and builds upon the positive findings of the meta-
analysis.
Collectively, these findings indicate that students not only have a more positive attitude about saving
money, educational aspirations, entrepreneurships, future orientation, civic attitude (i.e. social
consciousness for others and their communities) and rights attitude but also develop behaviours that
similarly reflect these new found attitudinal changes. Evidence of financial knowledge gain pertaining to
loan schemes, budgeting and profit calculation is also evident.
19
Students also showcase greater confidence in their interaction with others, and this is especially true for
girls. These positive changes, especially for girls, give strength to the positive gender attitudes observed
across case studies-with students being able to handle boy/girl relationships better in addition to boys
developing positive perceptions about girls. However, there are also some limitations to these positive
findings with difficulties ranging from poor context sensitivity to lack of adequate teaching materials,
among other challenges.
A limitation that may hinder these qualitative findings is that only 9 studies were used for the purpose of
analysis. Furthermore, only female participants were interviewed in one of the case studies16. Therefore,
these findings may not adequately reflect the global attitudinal and behavioural impact of Aflateen.
Further analyses are needed that include more studies. However, despite this limitation the overall
analysis shows a positive impact of Aflateen, adding weight to those findings that were significant in the
meta-analysis.
5. Sustainability Results The following section provides the overall findings of the global fsQCA study followed by the more
nuanced findings of additional case studies designed to investigate the strategies to sustain and expand
high quality Aflateen programmes. The last section of this chapter shines light on other aspects that
allow us to understand the factors that contribute to or hinder the sustainability of the programme,
derived from qualitative case studies on Aflateen.
fsQCA Results
The percentage of programmes that have continued is encouraging. A descriptive analysis of the first
four years of the programme found that 70% of partner organisations who started implementing
Aflateen continued to do so. However, only 32% have conducted the programme for at least 3 years.
Overall the study identified 3 pathways to Aflateen programme continuation and 4 pathways to growth.
Three strategies for sustaining (i.e. continuing) the Aflateen programmes have emerged from the fsQCA
analysis. The most common successful strategy, covering 20 cases, was to work with partners who have
experience with the primary school programme, Aflatoun, and to provide them with frequent support
from the Aflatoun Secretariat while ensuring high programme fidelity (Consistency = 0.81; Raw Coverage
= 0.28). This strategy was also the most likely to result in national integration, with four cases of national
integration. The other two strategies were for helping new partners sustain an Aflateen programme. One
strategy was to find strong funding support for implementing (and monitoring) a high-fidelity
programme that does not need frequent support from the Secretariat (Consistency = 0.80; Raw Coverage
= 0.13). The last strategy for new partners did not require funding but did require frequent contact with
the Secretariat and was supported by strong programme fidelity (Consistency = 0.83; Raw Coverage =
0.11). These two strategies for sustaining a new Aflateen programme suggest that funding and high
touch support can function as substitutes for each other.
16 This was the case for Tajikistan, with only female participants.
20
The analysis of the growth outcome identified four strategies to expand the reach of Aflateen
programmes. Only two of the four strategies were used to expand programmes that averaged more than
1,000 youth. One such strategy was to provide Secretariat services to partners who have experience in
implementing Aflatoun and are conducting high fidelity Aflateen programmes (Consistency = 0.79; Raw
Coverage = 0.22). This was also the only strategy that included programmes achieving national
integration (N = 2). Another strategy for growth in large programmes – both with and without
experience with Aflatoun – was to ensure that the partner had access to funding to implement a high-
fidelity programme17 (Consistency = 0.84; Raw Coverage = 0.18). A third strategy identified for growing
smaller programmes, was to provide high frequency Secretariat support and services to new partners
without funding (Consistency = 0.87; Raw Coverage = 0.11) – this strategy seemed to be high cost with a
low yield in terms of numbers of participants. The final strategy, which should be discouraged, was to
provide Secretariat services to low fidelity programmes without Aflatoun experience (Consistency = 0.88;
Raw Coverage = 0.13).18
More details can be found in the full published report available online from Aflatoun International.[23]
Other qualitative findings related to sustainability
The analysis of the qualitative studies on Aflateen revealed other emergent findings, prevalent in in this
programme. These issues – expected to influence the sustainability of the programme – include
challenges in the implementation of Aflateen (and how to overcome these), students’ reasons for
joining, and challenges related to the curriculum. Some of these studies were explicitly designed to
understand the growth and continuation pathways from the fsQCA study better.
Implementation of Aflateen
Effective implementation of Aflateen was impacted by a number of factors. For example although
Aflateen was implemented in many countries some students struggled to gain a positive outcome from
it. One teacher in Georgia said:
“…poor economic conditions and attitudes prevailing in community. These factors may
create challenges for the participants in their effort to be the agents of positive changes
in their communities.”
Other students struggled to meet the objectives of the programme. One teacher in Indonesia said:
“…lack of facility or understanding towards the program make the financial outcome
targeted has not achieved maximally.”
17 In this strategy the analysis showed that funding was the core condition while fidelity was a peripheral condition. 18 It should be noted that programmes can substitute core components with content participants are already receiving through the partners’ other, non-Aflateen, programmes.
21
As we found in a few different countries, school curricula are often overburdened. Aflateen-related
activities are therefore mostly extracurricular, which in turn “makes it hard to monitor in terms of quality
and in particular comprehensiveness” (PEDN in Uganda).
To be more effective, teachers in Nepal felt that more time needed to be allocated for the Aflateen
programmes, but they struggled to combine the programme with the rest of their classes:
“…need more time to be allocated to learn more from the Aflateen programme.
Teachers also separately identify time as a challenge for them to cope with normal
teaching and learning schedules as well as providing support to the students.”
The same issue exists for students, as they are sometimes equally busy. In Egypt, there was a clash of
Aflateen session timings with critical timings at schools, such as the start of the academic year, start of
the second half of the year, and examinations, leading youth to reduce their attendance or drop out of
the programme. A facilitator mentioned:
“Aflateen sessions start when school class times and timings for the private tutors
remain irregular. This poses a challenge to youth as sometimes the session timings clash
with their classes (especially youth in evening school shifts), and private tutors.”
The same was found in Indonesia. In addition to struggling to make time between class and the Aflateen
programme, facilitators and teachers also struggled to play the role of trainers during the programmes.
For example, in Georgia:
“…teachers found most challenging to make a shift from the role of a teacher to the
role of a trainer. Sometimes they started to dominate the group process… [they] tried
to make group discussions more disciplined.”
Another challenge to sustainable programming is the brain drain of teachers and facilitators. In Uganda,
we saw that skilled teachers are a scarce resource, because teachers often get promoted to different
schools, often from rural areas to urban areas. These human resource issues make continuation of
Aflateen in rural areas a continuous challenge.
Curriculum challenges and limitations related to the content
The analysis revealed several challenges with the Aflateen curriculum, such as some parts of the
curriculum being considered too complicated and irrelevant to certain contexts. For example, in Egypt:
“Parts on budgeting, work plans, developing proposals, high-budget enterprises were
specifically highlighted by the team as complicated and irrelevant sessions to the
available resources in the target villages and financial conditions of the youth.”
The qualitative analysis also revealed challenges to teaching across case studies. In some instances,
teachers did not bother to read the whole curriculum before teaching it as they considered it to be too
long. This was the case in Indonesia:
22
“According to the trainers and facilitators, the curriculum is really long and local
trainers hardly read everything from the curriculum.”
In some cases, teachers struggled with teaching certain topics such as those that were related to gender
and sexuality. These were considered taboo topics with some schools not being ready to implement
them. As noted earlier, one teacher in Indonesia said:
“…the case about gender difference and sexuality is the most sensitive thing to share
in their study level. Our norm strict us to be open discuss about that content”.
Reasons for joining and the corresponding challenges that resulted from joining
For programme continuation it is important that young people are interested in joining the programme –
because this means it is relevant and fun for them. Across case studies, reasons for joining Aflateen
varied. The majority of the reasons were oriented around social influence with friends or their teachers
influencing them to be part of the programme. One student in Georgia said:
“First my friends were not interested. I’m from village school and they told me why
bother to go to the town every week. Now they also would like to join the program to
see what makes me so motivated and involved.”
Another student in Kenya said that she played an important role in motivating her classmates to join:
“I talked to my classmates and some of the boys agreed to join.”
Some participants came to Aflateen with a desire to learn more about how they could help their
communities and improve the status of their villages, as well as being interested in learning more about
their rights to improve the quality of their life in school. One facilitator in Egypt said:
“They…hope that they will learn how to direct this passion and dynamic spirit to benefit
their communities and improve the overall status of their villages. This was coupled
with an interest to learn more about their rights and responsibilities, stemming from a
feeling that they are not living the quality of life they are hoping to and need to stand
up for themselves more, especially at schools, to face maltreatment by teachers and
school administrators.”
However, there were a number of challenges that impacted students’ desire and ability to join Aflateen.
For example, in some cases students weren’t interested in being part of the programme because of their
interest in other things. This resulted in a high dropout rate. One facilitator in Egypt said:
“…we were facing a difficulty to attract youth to join Aflateen and whole groups
dropped out at the beginning of the second part of the academic year….they had
different interests…”
However, there were instances wherein facilitators came up with interesting activities to attract
students’ to join Aflateen and keep their interest-like in the case of Egypt where a theatre group was
created as a means to transfer financial and social education.
23
“We were facing a difficulty to attract youth to join Aflateen and whole groups dropped
out at the beginning of the second part of the academic year, so… we form[ed] a theatre
group and started training them....”
Despite this, there were still some cases where youth were difficult to reach and convince as they were
already taking part in illicit activities such as drug use and gang fights. One teacher in Nepal said:
“…youth who really need social and financial education… are already into drugs and
into gang fights. It’s difficult to bring them under one room for training.”
Parents also played a huge role in impacting whether students joined the programme or not. In some
cases they were reluctant to enrol their children, especially the girls, as they felt that doing so would go
against the community traditions and brainwash their children. One facilitator in Egypt said:
“…the resistance of most parents to enrol their children in Aflateen, especially for girls
[is due to] Aflateen making girls over exposed in ways that contradict community
traditions.... they are afraid that programmes like Aflatoun International work like a
spy agent that brainwash the youth…”
Other programmes found a way to deal with this; the teachers in Georgia had a positive impact on
convincing parents to let their children participate in Aflateen:
“Trust towards the teachers determined that parents gave their agreement on pupils’
participation: ‘When I heard that her teacher invited her I let her attend, because we
like the teacher very much.’”
Future focus/improvements
A number of areas were suggested to be improved or focus upon in the future, such as the need for
additional material pertaining to financial education. One teacher in Indonesia said:
“…produce handy manuals for each module. In addition, child-friendly or youth-friendly
comics, visual aid materials and making use of informational technologies should be
produced for effective dissemination of Aflateen program and for the delivery of
sessions on social and financial education.”19
Language barriers can be an important challenge to the continuation or growth of the program, and the
Aflateen network. There were instances wherein teachers felt they needed materials that were in line
with specific languages and cater to both urban and rural domains. For example in India:
“There is a need to revise educational materials in terms of language. The language of
the manual is more urban in nature while the program is implemented in rural areas as
well.”
19 There is an online version of Aflateen called Aflateen Digital at Aflateen.org but it is not currently available in Bahasa.
24
Some in Tajikistan also voiced the need for more educational sessions for girls – which has been
established in a programme that combines Aflateen with sexual and reproductive health education for
adolescent girls20:
“Need more education sessions on financial and social, and also rights and
responsibilities for adolescent girls”
The Chinese partners are integrated into the global Aflatoun network. Even though their Aflateen
materials are available in Chinese, they feel a barrier to share experiences with peers outside of China.
Others emphasize a need to widen the applicability of the programme to target youth with disabilities
(Nepal):
“…come up with comprehensive financial literacy programs customized to meet the
needs of youth with all types of disabilities…”
Another potential area of improvement was the frequency of the programme. Students in India said they
wanted it to be delivered more often.
“It is recommended to boost this frequency to at least once a week so students do not
feel a break in the continuity of the program.”21
Some students also wanted there to be an awareness workshop for their parents to make them more
cognizant of a girl’s education. This was evident in Tajikistan:
“…have some awareness raising activity with their parents and explain them not to
isolate their adolescent daughters from education.”
Factors that contribute to the growth of Aflateen
In different countries and with different implementing partners, we see different strategies for scaling up
Aflateen. In the Ukraine, an enthusiastic civil society partner, The School of Equal Opportunities, has
allowed for growth of the programme, despite challenges and limitations. Mostly thanks to the teachers,
schools, students, and local school administrations, who are all very enthusiastic and supportive of the
initiative, the School managed to carry on and even expand the programme. The School keeps investing
in teachers to keep them engaged and interested in Aflateen, which was the key to success.
One of the most sustainable solutions for a program like Aflateen is to integrate it into the national
curriculum. In a number of countries, significant steps have been taken towards national integration.
Process tracing assessments show that, for example in Kenya, this was possible thanks to the
20 Aflatoun International together with Credit Suisse and Plan International UK conducted a desk review that examines to what degree, how and in what contexts the inclusion of financial education in interventions targeted at adolescent girls contribute to their economic empowerment. One of the main findings was that financial education combined with social and sexual and reproductive health education is most effective for adolescent girls. The full desk review is available here, and a policy brief was published that summarizes the main findings. 21 It is worth noting that many partners do implement Aflateen once per week and this is dependent upon the local partner.
25
investments of the financial regulators – they were the initiators of the integration process, therefore
ensuring their commitment and support. An important step in the process was a needs assessment, to
determine the gaps on financial education in Kenya, and securing buy-in from additional stakeholders.
National teachers’ organisations can also provide the impetus for national integration, as was the case
with the National Autonomous Secondary Education Union (SNAES) in Cameroon. Success factors were
the quality of the national and international training sessions, provided by Aflatoun International, and
the enthusiasm of the master trainers. Also the responsiveness and collaboration with the Ministry of
Secondary Education – and SNAES’s close involvement and monitoring facilitated the process and their
shared vision with Aflatoun International on quality education led to integration of Aflateen in the
national secondary education curriculum.
When comparing the situation in Peru with the process in Cameroon, we see interesting differences
between the two processes. In Peru, the partner Vision Solidaria never had a long-term strategic
planning for growth, but they were able to seize upon several opportunities, resulting in growth of the
program. In the case of SNAES, growth was planned and a strategy was crafted by key players in close
collaboration with the Secretariat long before the actual scaling happened. While both approaches were
successful in growing the programs— Vision Solidaria signed a MoU with the Ministry of Education on
April 5, 2016, and SNAES signed a contract for Aflateen’s integration into the national curriculum in
September 2015—both cases had very different processes.
A crucial factor in each of the three countries is timing: before the country goes through a revision of the
national educational curriculum, the stakeholders advocated for the inclusion of more financial
education and life skills in the curriculum. Also a multi-sectoral approach is recommended, as various
and diverse organisations, financial entities, curriculum experts and others are involved in the creation of
a comprehensive curriculum.22
Sustainability Summary
Two of the most important considerations that lead to sustainability, as measured through Aflateen’s
continuation and expansion, appear to be the support received from the Secretariat as well as the
fidelity of program implementation.
A number of implementation issues became evident through the analysis such as a need for more time
to teach the programme and the complexity and sometimes cultural sensitivity of the curriculum. Some
of these challenges call for better contextualization of the programme to the needs, capacity and social
norms of the target group – especially to make teachers feel comfortable delivering the content.
A challenge for scale up is when Aflateen is an extracurricular activity, because educational curricula are
too busy. As examples from Nepal (vulnerable out of school youth) and Uganda (brain drain in rural
22 More about processes towards national integration can be found in an Aflatoun International white paper by Shephard & Njengere, available here.
26
areas) show, it remains difficult to reach the most vulnerable children. Extra efforts need to be done to
reach them.
Students joined Aflateen for a number of reasons such as the influence of peers and the desire to make
their community better and learn more about their rights. However, a number of challenges seem to
impact the ability of youth to join Aflateen, such as a lack of interest, parent’s reluctance, and the school
timetable. Despite this, teachers and facilitators do try and come up with innovative solutions to attract
students to the programme.
Collectively, a number of challenges and suggestions for improvement are evident across case studies.
Taking account of these can help enhance the scope and width of Aflateen’s effectiveness and should be
considered when making future refinements.
6. Conclusions and Implications The Aflateen social and financial education programme has met with great demand internationally,
reaching more than 700,000 youth in 64 countries in 2015 and over 25,000 users of the Aflateen digital
learning platform. The growth of the programme since starting in 13 countries in 2011, has exceeded
youth participation targets by over 300% and country targets by 28%. The continued strength of the
programme can be explained in part by noteworthy positive impacts on youth as well as various
strategies to sustain and expand the program. Aflateen has a strong positive impact on young people’s
attitudes and behaviours. In short, it improves their capabilities.
Across the globe, Aflateen has positive effects on young people’s perspectives on the future (including
educational aspirations), both as a general orientation and specifically with regards to cultivating an
entrepreneurial spirit. Similar positive findings are also evident (through the qualitative analysis) for
students’ confidence/self-efficacy with students better able to interact with their peers, teachers and
parents. This is especially true for girls who are now more confident generally, including their
interactions with boys. These positive shifts in attitudes are not passive; they are accompanied by
improvements in corresponding behaviours, as was observed and testified by young people and adults.
Participants are more likely to manage their money responsibly and save for the future, as well as help
their parents in times of hardship. This resource management behaviour extends to non-monetary items.
The qualitative findings reveal that some students feel that the financial component of the curriculum is
not applicable to their life situation, due to economic hardships. These limitations highlight an
opportunity for curriculum refinements relating to financial education that are in line with each area’s
economic status to enhance the programme’s effectiveness. Qualitative evidence of financial knowledge
gain is apparent across case studies. Students feel that the lessons make them more knowledgeable
about loan schemes, budgeting, and calculating profit. The findings show that not all programmes are
able to cover this component in their respective curriculums and hint at a need to contextualize the
content.
Participating youth – both male and female – are more likely to have positive attitudes and behaviours
related to small-scale entrepreneurship activities. The qualitative findings add to this, as they show that
27
despite potential societal gender constraints prevalent in certain case studies, girls develop a desire to
start something of their own accord and become self-sufficient. At the same time there are some issues
that students’ face when enacting entrepreneurship behaviours, due to a lack of understanding about
businesses as well as poor support mechanisms. Therefore, creating additional guidelines that assist
participants in becoming more familiar with the ins and outs of entrepreneurships would enhance the
effectiveness of this component.
In addition to these outcomes, the positive signs of improvement in participants’ attitudes towards
women and human rights are encouraging even though they do not reach standard levels of statistical
significance in the meta-analysis. The qualitative analysis reinforces this positive trend, showing a
positive effect across case studies. Participants’ baseline attitudes on gender and rights are often
reinforced through family, friends, history, and culture. As such, it is encouraging that the global data
give us 90% confidence in a positive impact on these attitudes after a relatively short exposure to the
programme23. In particular, the qualitative analysis shows positive gender attitudes across case studies
with students better able to handle boy/girl relationships in addition to boys developing positive
perceptions about girls, with some exceptions which may be due to the topic being a taboo. The
qualitative data also find a positive impact on pro-social behaviour in the case studies, despite instances
of continued prejudice that are not easily altered through Aflateen. A similar observation is evident for
rights oriented attitudes and behaviour. Although cases show positive attitudinal and behavioural
changes for participants across case studies, there are also negative instances. Incorporating contextual
factors into programme design is key to improving this.
Other interesting findings emerged through the qualitative analysis: Youth joined Aflateen for a number
of different reasons such as the influence of peers, a desire to make their community better and learn
more about their rights, as well as to stand up to others. These reasons point to a genuine interest in the
programme and desire to be part of something positive. However, there are some emergent challenges
that impacted student’s ability to join -such as a lack of interest, time, or their parents’ resistance. There
are also a number of implementation issues that impact programme delivery by teachers and
facilitators. Some teachers and facilitators voice a need for more time to learn and then teach the
programme, as well as assistance in learning to facilitate young people’s discussions of such varied
topics. A number of potential concerns surrounding the curriculum are raised such as it having parts
that are too complicated, irrelevant, or culturally sensitive in certain contexts. Careful local refinements
to the curriculum are needed to be contextually sensitive. Many respondents also expressed a desire for
additional resources, such as visual aids. Through improved contextualization and the addition of
supporting tools as suggested, Aflateen may reach even more young people with even greater impact.
23 An analysis of the impact of the Aflateen Digital curriculum was also conducted finding impacts from exposure to the rights section of the online curriculum, that the planning section has an impact on entrepreneurship, and that completion of the money section affects reported savings. Source: Bearden, G. (2015). An Analysis of the Effects of Aflateen Digital on Social & Financial Outcomes for Youth and Platform Attrition. Aflatoun International Working Paper 2015. Accessible here.
28
Although the long-term sustainability of the programme has yet to be seen, analysis of the first four to
five years of the programme are positive with over 70% of partner organisations who started
implementing Aflateen continuing to do so. The three identified strategies for sustaining Aflateen
programmes indicate the importance of programme fidelity for sustainability. This highlights the
importance of setting an expectation for high fidelity among partners, assisting in strategies to achieve
increased fidelity, and ensuring that a basic monitoring system is in place. It is also notable that funding
and high levels of Secretariat support can function as substitutes for each other in the pathways to
sustainability for new Aflateen partners. This suggests two different funding models, one in which
partner funding is the focus and another in which funding for increased support functions from the
Secretariat and other partners is the focus. For partners with experience implementing Aflatoun’s
primary school curriculum, it is still important that the Secretariat continue to provide support and
monitor fidelity – as this was the only pathway to sustainability for partners who had previous
experience with Aflatoun.
The growth of Aflateen has been the result of recruitment of new partners, encouraging current Aflatoun
partners to implement Aflateen, and the expansion of existing programs. The program’s midterm
evaluation found that one of the most cited reasons that organisations started implementing Aflateen
was its flexibility and ability to be merged with their existing programmes as well as its methodology and
content.[16] In the fsQCA analysis which identified four strategies to increase the number of Aflateen
participants, only two of the four strategies were used to expand programmes that averaged more than
1,000 youth. As was the case with continuation, these two strategies again highlighted a “high-touch”
support (high service intensity and frequent Secretariat contact) strategy and a “low-touch” funding
strategy. Also in line with the continuation findings, these two strategies both highlighted the
importance of high fidelity as it was a core condition of the high support strategy and a peripheral
condition of the funding strategy. The other two strategies seem suboptimal. One requires low fidelity,
which is not desirable from a quality control stand-point. While the other requires high levels of support,
uncertain fidelity, and seems to only assist in growing small-scale programs.
At the core of the model for implementing Aflateen is the partnership between the international
Secretariat, intentional and local civil society, educational institutions, and financial institutions. At times
when funding may be scarce, partnerships have enabled Aflateen programmes to continue. Civil society
organisations with previous experience implementing Aflatoun (CSFE for primary school age-groups) are
particularly resilient in continuing Aflateen with strong technical support from the Secretariat despite
lower levels of funding as is highlighted in the fsQCA pathway on continuation for previous Aflatoun
partners. Through civil society’s partnership with education institutions additional sustainability can be
achieved, as was highlighted in the Ukraine case study in The School of Equal Opportunities, where
schools and teachers that understand the value of the programme and its impacts on young people
provided space and time to continue the programme with the support of an enthusiastic civil society
partner. National teachers’ organisations can also provide the impetus for national integration to enable
greater sustainability and scale, as was the case with the National Autonomous Secondary Education
Union (SNAES) in Cameroon. Furthermore, linking Aflateen partners and programmes with Central Banks
and other financial institution efforts to promote financial inclusion and education has also enabled
29
programmes to obtain additional support (both political and financial) to continue programmes.
Partnerships with programmes and policies linked to entrepreneurship also show promise for supporting
Aflateen programmes. This can be seen through the case studies in Nigeria and Peru which note
experiences of linking Aflateen implementation with entrepreneurship – and the interest therein. This
interest can be connected to the findings of our meta-analysis showing Aflateen’s positive impacts on
entrepreneurial attitudes and behaviour.
Implications for Practice
Aflatoun International
Based on the findings above some recommendations for The Aflatoun International Secretariat follow:
a. Create advocacy materials for partners to use that highlight Aflateen’s impact with a
special focus on entrepreneurship.
b. Continue to place focus on its impact on behaviour change and shifting in young people’s
world view and less on traditional knowledge-based financial literacy outcomes.
c. Conduct a cost-benefit analysis to determine if increasing technical support capacity is
more cost effective than direct fundraising.
d. Incentivize higher fidelity programmes by sharing its importance to success, requiring
yearly submission of fidelity information, and publishing fidelity information.
e. Prioritize technical support of Aflateen programmes by partners with experience
implementing the primary school Aflatoun programme.
f. Enhance connectivity between partners by promoting and allocating resources to create
Aflateen specific discussions.
Partner Organisations
We recommend current and prospective Aflateen partners consider the following recommendations as
appropriate for their context:
a. Identify opportunities to link Aflateen programmes to entrepreneurship initiatives,
funding, and policies in the local context.
b. Use the evidence of Aflateen’s impact on future orientation and savings behaviour to
build partnerships with Central Banks and financial institutions.
c. Strengthen partnerships with teachers and teachers unions to sustain and expand the
program. Providing training in the active learning methods of Aflateen can assist.
d. Conduct needs assessments to understand the needs and capacity of the target groups,
to avoid challenges with content that is too complicated, or not challenging enough, and
to contextualize content.
e. Work to ensure the most vulnerable children are reached, also in rural and remote
areas.
f. Work to improve and track the fidelity of your Aflateen programme because
programmes with higher fidelity have more impact, sustainability, and growth.
g. Make use of the Aflatoun International Partner Portal to share best practices with other
Aflateen partners and to ask questions.
30
h. If partners graduate programmes (i.e. Aflateen after Aflatoun) they might need more
complex curricula in order to allow for greater gains in learning.
Other Organizations
a. Relevant multinational organisations and national ministries should consider how
Aflateen can support their strategies in meeting the Sustainable Development Goals.
Specifically how Aflateen contributes to:
a. Goal 4 targets on skills for entrepreneurship and education on human rights and
gender equity;
b. Goal 5 targets on empowering women economically and eliminating
discrimination against women;
c. Goal 8 targets on reducing the number of youth not in employment, education
or training and increasing the protection of labour rights;24
d. And Goal 17 on enhancing North-South, South-South and triangular cooperation
and capacity development.
b. Teacher training unions and organisations should consider how Aflatoun and Aflateen
partners can provide training on active learning methods - for teachers - that are
applicable throughout their curriculum.
c. Funders should focus their support on the technical service provision and capacity
building roles of the Secretariat and experienced partners while using implementation
focused funding to encourage new organisations to start implementing Aflateen.
Implications for Research
Aflatoun International, Aflateen partner organizations, and external researchers should focus on both
confirming and expanding the current Aflateen evidence base in the following ways:
a. Impact evaluations of Aflateen should measure future orientation, resource management, and entrepreneurship – especially when conducted in new settings.
b. Future evaluations should also measure gender attitudes and rights orientation to add additional data to confirm the positive trends highlighted herein.
c. Future studies should use the reported effect sizes to determine sufficient sample sizes prior to conducting evaluations. Many of the current studies of Aflateen suffered from small sample sizes that may have been unable to detect important effects.
d. An effort should be made to compile a database of all Aflateen Surveys in order to conduct individual level interaction analysis using demographics and outcomes.
e. Future studies should investigate the interaction between future orientation and resource management behaviour (especially savings) as well as between entrepreneurship attitudes and behaviours.
f. Conduct longitudinal evaluations to determine if outcomes are sustained and to examine how and to what extent Aflateen sustainably impacts quality of life or quality of education.
24 The Aflateen curriculum contains learning modules on labour rights.
31
Summary Conclusions
The Aflateen programme has expanded throughout the globe in less than 5 years touching the lives of
millions of young people directly and indirectly. The combined analysis of evaluations through the globe
show a strong impact on future orientation, resource management behaviour (especially savings), and
both entrepreneurial attitudes and behaviours. The programme also shows positive trends regarding its
impact on gender attitudes and rights orientation although more data is needed. The average size of
these effects is larger than the average educational intervention.[28; 29] These positive findings are further
enhanced through the positive effect observed in the qualitative analysis across case studies. This
indicates that the impact of Aflateen is quite robust. The network should continue to build on this
success in improving the outlook and entrepreneurial spirt of young people while also focusing on key
outcomes that are unique to their country and context.
The research conducted herein has identified pathways to continue and expand Aflateen under scenarios
of high and low funding thanks to the constant learning and innovation of the Aflatoun/Aflateen
network. The importance of programme fidelity, technical support and capacity building, and partnership
are clear throughout the findings. As a result of its research and partner feedback, Aflatoun International
has already begun restructuring its way of working to better enhance and support these features as part
of its 2016-2020 Strategy. Armed with a breadth of both evidence and experience, it is our hope that the
Aflateen network continues, refines, and expands its strong positive influence on young people.
7. References
1. Barro R. J. & J. W. Lee. 2013. A new data set of educational attainment in the world, 1950–2010. J. Dev. Econ. 104: 184-198.
2. UNESCO. 2015. Education for All 2000-2015: achievements and challenges. EFA Global Monitoring Report 2015.
3. Schwab K. 2016. The fourth industrial revolution. In World Economic Forum Geneva.
4. Twenge J. M., N. T. Carter & W. K. Campbell. 2015. Time period, generational, and age differences in tolerance for controversial beliefs and lifestyles in the United States, 1972–2012. Social Forces. 94: 379-399.
5. ILO. 2016. World Employment and Social Outlook 2016: Trends for youth.
6. The World Bank. 2014. Global Financial Inclusion Database.
7. Supanantaroek S., R. Lensink & N. Hansen. 2016. The Impact of Social and Financial Education on Savings Attitudes and Behavior Among Primary School Children in Uganda. Evaluation Review.
32
8. UNICEF. 2012. Global Evaluation of Life Skills Education Programmes.
9. Child & Youth Finance International. 2016. Economic Citizenship Education for Children and Youth.
10. Berry J., D. Karlan & M. Pradhan. 2015. The Impact of Financial Education for Youth in Ghana. National Bureau of Economic Research Working Paper Series. No. 21068.
11. Shephard D. D., Y. Kaneza & P. Moclair. 2015. Financial Education for Children: What Curriculum? Which Methods? A Cluster Randomized Controlled Trial of Social and Financial Education in Rwanda. In WERA.
12. The Brookings Institution. 2016. Aflatoun International - Scaling social and financial education through a global franchise.
13. Atkinson A. & F. Messy. 2013. Promoting Financial Inclusion through Financial Education.
14. OECD. 2005. Recommendation on Principles and Good Practices for Financial Education and Awareness. Recommendation of the Council.
15. Njengere D. & D. Shephard. 2015. Social and Financial Education: Documenting National Cases and Processes of Curriculum Integration. Aflatoun Working Paper.
16. Ramirez R. & G. Kora. 2013. Utilization Focused Developmental Evaluation of the Aflateen Secretariat Support. See https://aflatoun.box.com/v/AflateenMidtermEvaluationVol1
17. Aflatoun International. 2016. Annual Report 2015.
18. Karimli, L., McKay M.M., Shephard, D., Kurtz, J., Komilzoda, S., Te Kaat, A. (2015) Aflateen+ Impact Evaluation Endline Report, New York, NY: MercyCorps, Aflatoun International. See https://aflatoun.box.com/v/TajikistanRCTBrief2016
19. de Mel S., D. McKenzie & C. Woodruff. 2014. What Generates Growth in Microenterprises? Experimental Evidence on Capital, Labor and Training.
20. Card D., J. Kluve & A. Weber. 2010. Active labour market policy evaluations: A meta‐analysis. The Economic Journal. 120: F452-F477.
21. Steinberg L., S. Graham, L. O’Brien et al. 2009. Age differences in future orientation and delay discounting. Child Dev. 80: 28-44.
22. Fernandes D., J. G. Lynch Jr & R. G. Netemeyer. 2014. Financial literacy, financial education, and downstream financial behaviors. Management Science. 60: 1861-1883.
33
23. Meuer J., A. Ellersiek, C. Rupietta et al. 2016. The MasterCard Foundation Final Aflateen Evaluation: Global Report. In To Continue and Grow: Final Evaluation of the MasterCard Foundation Partnership with Aflatoun International on Youth Social and Financial Education. te Kaat A., Ed.: 8-47. Aflatoun International. Amsterdam.
24. Ragin C. C. 2008. Redesigning Social Inquiry: Fuzzy Sets and Beyond. Wiley Online Library.
25. Schneider C. Q. & C. Wagemann. 2012. Set-Theoretic Methods for the Social Sciences: A Guide to Qualitative Comparative Analysis. Cambridge University Press.
26. Collier D. 2011. Understanding process tracing. PS: Political Science & Politics. 44: 823-830.
27. O'Prey L. & D. D. Shephard. 2014. Financial Education for Children and Youth: A Systematic Review and Meta-analysis. Aflatoun Working Paper. : July 21, 2015.
28. Hattie J. 2009. Visible Learning a Synthesis of Over 800 Meta-Analyses Relating to Achievement.
29. Hattie J. 1992. Measuring the Effects of Schooling. Australian Journal of Education. 36: 5-13.
34
8. Annex Table 2: Meta-Analysis Study Descriptions
Study N
Study Type
Age (M)
Partner Notes
Bulgaria 458 Pre / Post
16.5 Integra 4-month study using the Aflateen survey.
China 161 Pre / Post
19.0 Be Better 2-month study among vocational students in Shanghai. Report available here.
Honduras 29 Pre / Post
15.6 ChildFund 7-month study following 29 matched youth w/ Aflateen Survey. The programme took place in community locations.25
Saudi Arabia
104 Pre / Post
15.0 AUDI, Wa3i, Al Nahda
10-day training program, a different survey was used. The sample included participants in Riyadh and the Eastern Province at the end of 2014.
Saudi Arabia #2
30 Pre / Post
15.6 AUDI, Wa3i 10-day training programme implemented in the Eastern Province in 2015 among a different cohort of youth using the Aflateen survey.
Tajikistan 1,269 RCT 16.7 Mercy Corps 18-month cluster randomized controlled trial combining Aflateen with two modules on sexual and reproductive health. All participants were female. Brief available here.
The Gambia
336 Pre / Post
19.026 ChildFund 24-month study using the Aflateen survey. The Aflateen programme is conducted after class among in-school and out-of-school youth for an average of 6 hours per week. Full report available here, and summary here.
Zimbabwe 59 Pre / Post
16.3 Junior Achievement & Plan International
10-month study using the Aflateen survey. The programme took place as an extra-curricular activity at secondary schools in Mutasa. Report available here.
25 The full study included 377 students; however, the original study author focused on the 29 matched pairs for the evaluation. 26 The mean age of participants as estimated.
35
Meta-Analysis Forest Plots
Positive Impacts
Figure 2: Future Orientation Forest Plot (Random Effects w/ REML)
Figure 3: Resource Management Behaviour Forest Plot (Random Effects w/ REML)
36
Figure 4: Entrepreneurial Attitudes Forest Plot (Random Effects w/ REML)
Figure 5: Entrepreneurial Behaviour Forest Plot (Random Effects w/ REML)
37
Positive Trends Figure 6: Gender Attitudes Forest Plot (Random Effects w/ REML)
Figure 7: Rights Orientation Forest Plot (Random Effects w/ REML)
*Detailed forest plots of the non-significant findings are available upon request. Summary statistics for
all outcomes (significant and not significant) are in Table 1 above.
38
Qualitative and Sustainability studies
Table 3: Qualitative and Sustainability Studies Used for Analysis
Country Author Title Link
Qualitative studies Egypt Wafeya El-Shinnawy Research on the Aflateen Programme:
Experiences from Rural Egypt (Internal document)
Full report available here & summary available here.
Indonesia CLDS Indonesia Team Aflateen Programme in the Context of Indonesia: Challenges and Opportunities
Full report available here.
Kenya Action for Child Development Trust
Aflateen Evaluation Final Report Full report available here
Nepal Lachhindra Maharjan (Aflatoun, CWIN and UNICEF)
Aflateen Programme in Context of Nepal: Challenges and Opportunities
Full report available here
India Kishore Darak Impact Analysis- Aflateen Social and Financial Education Programme Aflateen: Experiences from India
Full report available here and summary available here
Tajikistan Mercy Corps Tajikistan Focus Group Discussion for Aflateen program among adolescent schoolgirls in Hisor and Mastcho district
In press.
The Gambia
Andrew Magunda Evaluation of the Aflateen Programme in The Gambia
Full report available here and summary available here
Congo Venanr Bashige (Action pour de development pour la femme et enfant (ADEFE) and Aflatoun International)
Rapport d’evaluation a mi-parcour du programme Aflateen
Full report available here
Georgia Medesa Despothashvili (Stichting Child Savings International and Foundation for the Development of Human Resources)
Qualitative Evaluation of Aflateen in Georgia
Full report available here
Sustainability (that have not mentioned above yet) Multiple countries
Dr. Johannes Meuer, Dr. Annkathrin Ellersiek, Dr. Christian Rupietta & Dr. Katherine Caves
Aflateen Final Evaluation 2015-2016: Continuation and growth of the Aflateen program: A Global Study
Full report available here
Cameroon Protais Bidzogo Onguene, Emeran Tsanga Guy, Flora Noumi Nzouawa and Prof. Joseph Bomda
Study on the integration process of the Aflatoun curricula in Cameroon
Full report available here and summary available here
Kenya John Masolo (with input from David Njengere, KICD, Aukje te Kaat and Rediet Abiey, Aflatoun International
Experiences, challenges and lessons learnt from the process of mainstreaming financial literacy into the primary and secondary school curricula in Kenya
Full report available here