bala janaagraha impact assessment,...
TRANSCRIPT
Bala Janaagraha Impact Assessment, 2014-2015Janaagraha Center for Citizenship and Democracy
Bangalore, 2015
1
Program Overview
The Bala Janaagraha program is designed to educating the futurecitizens about roles and responsibilities of citizenship, theprocesses and institutions of governance
The Bala Janaagraha Program runs from June/July toFebruary/March
The Bala Janaagraha program has all categories of schools onboard (Government, Private-Aided and Private)
The classes are delivered in the medium of English, Hindi andregional languages such as Kannada and Malayalam
The delivery of the program is managed by a Team of 54Facilitators spread across the 24 cities
2
Program Overview
Increase (in percentage) in Bala Janaagraha students between academic years: 118
Increase (in percentage) in the number of Bala Janaagraha schools between academic years: 108
Increase (in percentage) in the number of city outreach between academic years: 118
Bala Janaagraha Program 2013-2014 2014-2015
Total No. of Cities 11 24
Total No. of Schools 233 485
Total No. of Students 16, 439 36, 000
3
Donor Partnerships for Bala Janaagraha Program
8
4 4
2 2 2
1 1
0
1
2
3
4
5
6
7
8
9
Dell HSBC Pirojsha GodrejFoundation
Mindtree BPCL GMR Group Sharekhan TPG Capital
Year
s o
f P
artn
ersh
ip
Donor Partnership for Bala Janaagraha Program
Donors Partnership (In Years)
Donors
4
The Bala Janaagraha Program has expanded largely due to the financial support from like-
minded donors and institutions, who believe in the work areas of Janaagraha and have
partnered with Janaagraha from program inception
Theory of Change: Bala JanaagrahaActivity/Input Output Intermediate Outcomes Long-term Outcomes
Ob
jecti
ves
-Bala Janaagraha Resource Book.
-12 Curriculum Sessions delivered by
the Facilitators
-4 E-module sessions
-Increase in Civic Literacy Score
-Increase in Civic Awareness and
Civic Engagement
-Civic Fest
-Children actively take part in the development of
the neighborhood/city. E.g. Cleanliness Drives,
Planting Trees, Raising Awareness
-Children have adequate knowledge of the urban
governance that directly impacts the quality of
lives. E.g., Going to Councilors to lodge
complaints about public infrastructure
failure/repairs
-Children’s own attitude towards the city,
community and the Government is changed and
they influence the attitude of their family members
Active Citizenship
Ind
icato
rs
# of Bala Janaagraha Resource books,
translated into multiple languages, and
provided (for free) to the students
# of Facilitators delivering the
curriculum sessions
# of computer games delivered either
through individually handled computers
or as a classroom activity
-Percentage change in civic literacy
scores between pre/post surveys
-Photos taken by students in the
Bala Janaagraha Photo Project
-Interview excerpts from FGD
-Civic Project
-% increase in Civic Literacy Scores
-Increased involvement of Bala Janaagraha students
in civic campaigns
Increase in citizens engagement beyond
voting
Data
So
urc
e
-Project MIS
-Pre/Post Survey (Civic Literacy
Test )
-Focus Group Discussions
-Bala Janaagraha Photography
Project
-# of Civic Campaigns.
-I Change My Street/I Change My City
-Future Civic Projects
-Updated Student Information Database
-Updated Student Information
Database
-I Change My City website
-Janaagraha-Brown Citizenship Index
Ass
um
pti
on
s
-Schools allow Facilitators to hold the
Bala Janaagraha Curriculum Sessions.
-Schools have functional desktop/laptop
to deliver the E-Module sessions.
-Monitoring whether students are taking the test
without help from the friends/peers.
-Students are supported by friends/families in
raising awareness
-Updated information on civic campaigns and
volunteers involved in the campaign
Schools continue to take part in the
Bala Janaagraha Program and allow the
Janaagraha Research Team to use
quantitative and qualitative methods to
conduct Impact Assessment.
-Bala Janaagraha Resource Books
become available in multiple languages.
More Schools are encouraged to join
the Bala Janaagraha program.
-Integration of Bala Janaagraha database
with the institutional database that also
is connected to I Change My City,
allowing for longitudinal monitoring of
existing Bala Janaagrahis/alumnus of the
program
5
Impact Assessment : Different Variants
Bala Janaagraha
Program (2014-2015)
School-level Data Collection
Pre-Survey Civic Literacy Test
Pre
-Pro
gram
Sta
ge
Bala Janaagraha
Photography
Project
Post-Survey Civic Literacy Test
Focus Group Discussions
Po
st-Pro
gram Stage
Impact Assessment Research Methodology
School-level Data Collection Quantitative
Pre/Post Civic Literacy Test Quantitative
Focus Group Discussions Qualitative
Bala Janaagraha Photography Project Qualitative
6
Pre/Post Civic Literacy Test: SurveyBrief Excerpts
7
Pre/Post Civic Literacy Test: Respondent Mix
Population Sample Pre Civic Literacy Test Post Civic Literacy Test
Total # of Respondents (English) 24, 465 19, 355
Total # of Respondents (Kannada) 1,133 238
Total # of Respondents (Hindi) 629 867
Total # of Respondents 26,227 20,460
8
• For the Impact Assessment, we include all respondents who took part in the Pre/Post Civic Literacy Test. • The total sample size(n) for Impact Assessment =20,240
*Decline in # between pre/post surveys can be attributed to school dropouts, absentee students, etc
Themes for the Civic Literacy Test
9
Civic Knowledge
• Knowledge of three levels of Government; Local Government of the City; City Planning
• Responsibilities of the Local Government; Local Service Delivery Agencies; Civic Responsibilities & Citizenship Rights; Urban Planning themes and challenges
Civic Engagement
• Participating in civic activities
• Addressing civic problems
Civic Awareness
• Knowledge of civic issues specific to school locality and neighborhood
Key Findings: Survey
10
Pre/Post Literacy Test Scores
Schools/Cities Pre-Survey Test Score
(In Percentage)
Post-Survey Test Score
(In Percentage)
All Schools/Cities 39% 57%
11
Results: Civic Literacy Scores
*
12
0
10
20
30
40
50
60
Score
s (I
n P
erc
enta
ge)
Pre/Post Civic Literacy Test Scores-All Cities/Schools
Civic Literacy Scores (In Percentage): 2014-2015
Pre-Survey Test Score
Post-Survey Test Score
Results: Civic Literacy Scores
*
13
0
10
20
30
40
50
60
Score
s (I
n P
erc
enta
ge)
Pre/Post Civic Literacy Test Scores for All Cities/Schools
Civic Literacy Scores (In Percentage): 2014-2015
Pre-Survey Test Score
Post-Survey Test ScoreIncrease in civic
literacy scores by
18% (23points)
Results: Civic Literacy Scores
*
14
Civic Literacy Scores from Pre to Post Civic Literacy Test, increased in 19 out of 22
cities that took part in the pre/post surveys
*Schools from Bhubaneshwar and Ludhiana did not do pre/post surveys and Ludhiana did not take part in post surveys
0
10
20
30
40
50
60
70
80
AHM BHO BLR CHA CHE DEH DEL HYD JAI JOD KAN KOC KOL LUC MUM NAG PAT PUN RAI RAN TVM UDA
Sco
res
in P
erc
en
tage
Pre/Post Literacy Scores by Cities
Pre-Test Scores Post-Test Scores
Results: Civic Literacy Scores
*
15
Three cities (Ahmedabad, Kanpur and Kolkata) witnessed decline in Civic Literacy Scores in Post-Survey, marginally
Schools with more students scored lower than other schools
Post-surveys were started by students, but left incomplete (Kolkata, Kanpur)
*Schools from Bhubaneshwar and Ludhiana did not do pre/post surveys and Ludhiana did not take part in post surveys
0
10
20
30
40
50
60
70
80
AHM BHO BLR CHA CHE DEH DEL HYD JAI JOD KAN KOC KOL LUC MUM NAG PAT PUN RAI RAN TVM UDA
Sco
res
in P
erc
en
tage
Pre/Post Literacy Scores by Cities
Pre-Test Scores Post-Test Scores
3 cities witnessed decline in score
Results: Civic Engagement*Civic Activities Pre-Survey Post-Survey
Yes, I have
participated
No, I have not
participated
Yes, I have
participated
No, I have
not
participated
Save Water Campaign 38% 62% 61% 39%
Creating awareness on segregation of dry and wet waste 18% 82% 46% 54%
Save tree campaign 50% 50% 73% 27%
Cleaning the school campus 53% 47% 73% 27%
Safe and environment friendly festival (e.g., Diwali) 46% 55% 56% 44%
Traffic Police Day celebration 13% 87% 24% 76%
Demonstration of Rain Water Harvesting 25% 75% 43% 57%
Save electricity campaign/Switch off light campaign 41% 59% 59% 41%
Know your city campaign 24% 76% 50% 50%
16
*Respondents could select multiple activities
Results: Civic Engagement*Civic Activities Pre-Survey Post-Survey
Yes, I have
participated
No, I have not
participated
Yes, I have
participated
No, I have
not
participated
Save Water Campaign 38% 62% 61% 39%
Creating awareness on segregation of dry and wet waste 18% 82% 46% 54%
Save tree campaign 50% 50% 73% 27%
Cleaning the school campus 53% 47% 73% 27%
Safe and environment friendly festival (e.g., Diwali) 46% 55% 56% 44%
Traffic Police Day celebration 13% 87% 24% 76%
Demonstration of Rain Water Harvesting 25% 75% 43% 57%
Save electricity campaign/Switch off light campaign 41% 59% 59% 41%
Know your city campaign 24% 76% 50% 50%
17
28% increase in generating
awareness on segregation
of dry and wet waste
*Respondents could select multiple activities
Results: Civic EngagementCivic Activities Pre-Survey Post-Survey
Yes, I have
participated
No, I have not
participated
Yes, I have
participated
No, I have
not
participated
Save Water Campaign 38% 62% 61% 39%
Creating awareness on segregation of dry and wet waste 18% 82% 46% 54%
Save tree campaign 50% 50% 73% 27%
Cleaning the school campus 53% 47% 73% 27%
Safe and environment friendly festival (e.g., Diwali) 46% 55% 56% 44%
Traffic Police Day celebration 13% 87% 24% 76%
Demonstration of Rain Water Harvesting 25% 75% 43% 57%
Save electricity campaign/Switch off light campaign 41% 59% 59% 41%
Know your city campaign 24% 76% 50% 50%
18
Overall increase in different forms of Civic Engagement through different activities
between the pre and post survey.
Top three civic activities that witnessed greater student engagement: Save Tree
Campaign, Cleaning the School Campus and Save Water Campaign (which
are highlighted in the table)
*Respondents could select multiple activities
Results: Civic Engagement*
Visiting Civic Authorities Pre-Survey Post-Survey
Yes, I have
visited
No, I have
not visited
Yes, I have
visited
No, I have
not visited
Corporator of your ward 19% 81% 50% 50%
Police Station 32% 68% 40% 60%
Local Ward Office 17% 83% 39% 61%
Local Water Supply Office 25% 75% 40% 60%
Electricity Board 37% 66% 57% 43%
19
*All responses that were left blank were captured under the “no” category of responses
**Respondents were asked to identify one issue, but they identified multiple issues
Results: Civic Engagement
Visiting Civic Authorities Pre-Survey Post-Survey
Yes, I have
visited
No, I have
not visited
Yes, I have
visited
No, I have
not visited
Corporator of your ward 19% 81% 50% 50%
Police Station 32% 68% 40% 60%
Local Ward Office 17% 83% 39% 61%
Local Water Supply Office 25% 75% 40% 60%
Electricity Board 37% 66% 57% 43%
20
22% increase in the
number of visits to the
local ward office
• During the post-survey, greater percentage of students seem to have visited local
authorities/agencies post-survey
• Visiting the Local Ward Office witnessed the largest increase (22%) between pre/post
surveys
*All responses that were left blank were captured under the “no” category of responses
Results-Civic Awareness
Civic Issues Pre-Survey Post-Survey
Garbage Accumulation 26% 43%
Bad Roads 13% 32%
Frequent Power Cuts 6% 15%
Lack of Continuous Water Supply 5% 14%
Pollution 22% 29%
21
*Respondents identified more than one issue
Civic awareness increased in the post-survey period, since more students responded to the question on the main civic issues affecting their school locality
Garbage accumulation continued to be the top most concern for student in the pre and post survey
Key Findings: Focus Group Discussions
22
Focus Group Discussion Themes
I. Development of the City: Sustainable development will address
the future problems of a city .
Students received support from the Schools and Corporators to
conduct projects on sustainable development. In Pune procuring
support & cooperation was much easier than in Bangalore and Mumbai.
In Bangalore, the Corporator worked only due to unprecedented
pressure created by the Students.
II. Program Development: Bringing in real life examples and/or
have subject matter speakers with practical exposure make the
learning process much easier and more interesting.
Students suggests screening movies/short stories so as to aid the
learning process
23
Focus Group Discussions
III. Civic Engagement: Students continue generating awareness within
the larger community
In Pune, current group of Bala Janaagraha students started a tree
planting project that also involved 6th graders and 7th graders.
The intensions were twofold: First, engage 6th and 7th graders in civic
campaigns. Second, current 8th graders wanted to serve as potential
mentors for future Bala Janaagraha students.
IV. Program Learning: Greater awareness on themes of urban
planning, technology, maintaining and improving city life to behavioral such
as learning to converse with politicians and connecting with the local
people and helping people
24
Key Findings: The Bala Janaagraha Photography Project
25
Bangalore A Child’s View: Key Themes/Findings
26
Majority of children felt that they liked the parks in Bangalore. Some mentioned they enjoyed iconic green spaces in Bangalore, including LalBagh and Cubbon Park.
Bala Janaagrahis were worried about green spaces that had been destroyed or dirtied with garbage, and said they needed more parks.
Bangalore A Child’s View: Key Themes/Findings
27
.
Students identified electric hazards as one of the main challenges
Reasons for concern: loose electric wires and open transformers in parks and roads; power shortages affecting families and the economy
Bangalore A Child’s View: Key Themes/Findings
28
.
Students identified electric hazards as one of the main challenges
Reasons for concern: loose electric wires and open transformers in parks and roads; power shortages affecting families and the economy
Garbage: Every child who participated in the project is concerned about the garbage in Bangalore due to its implications on public health and environment.
Solutions: Government needs to do better job at dealing with garbage; People must take responsibility and clean up themselves
Bangalore A Child’s View: Key Themes/Findings
29
Children felt that the government and the citizens should grow more trees, plants and flowers. They were concerned about the impact of garbage, pollution on nature, health of Bangalorean and animals• Majority expressed concerns about wastage of environmental
resources, such as water
Bangalore A Child’s View: Key Themes/Findings
30
Children were concerned with the conditions of roads and footpaths in the city. Students noticed potholes, uneven roads and other things that could cause motor accidents.• Broken footpaths, open drainage and other obstacles make it hard for people to walk
around and could injure pedestrians
Bangalore A Child’s View: Key Themes/Findings
31
Students noted that roads need to be planned better with moresafety measures. Others criticized rash driving and said trafficissues create problems for pedestrians, including children andthe elderly.
Conclusion
32
Conclusion: Survey
From the pre to post survey, the civic literacy scores improved
by 18% (Score difference: 23 points)
The difference in scores indicate the impact of the Program in the improvement
of the civic learning among students of the Program.
Civic engagement and civic awareness for Bala Janaagraha
students increased in post survey
33
Conclusion: Focus Group Discussions & Photography Project
In Focus Group Discussions students’ expressed willingness to engage with
the program post completion. They continue to participate in various civic
activities (e.g., save tree campaigns; traffic police day celebrations)
Students engage with family and friends in raising civic awareness. But they
often face resistance from their families, when they try to participate in civic
activities
The Bala Janaagraha Photography Project reveal that garbage, environment,
parks and street/footpaths, were among the major themes of concern for
urban planning and development.
The solutions offered by a Bala Janaagraha students are more policy oriented than non
Bala Janaagraha students
Bala Janaagrahis are more willing to be participants of civic activities than non-Bala
Janaagraha students
34
Future Road map for Impact Assessment
Organizing workshops for the facilitators to inform on why/how Impact Assessment matters for the purposes of evaluating the Bala Janaagraha program evaluations
Surveys to be conducted by external survey agency that allows to remove bias in execution of surveys
Streamlining the surveying process with redesigning the survey and using Tabs to conduct surveys. By doing so, the analysis can be done more effectively & efficiently
Focus on the use of qualitative tools, including Focus Group Discussions and Structured Interviews to study civic learning among Bala Janaagraha students
35
Annexure: Methodology
36
Methodology- Survey
Sample Selection: All students who are part of the Bala Janaagraha program
take part in survey. However, only those students who have taken part in the
pre/post survey are included for Impact Assessment
Students were tracked using the Student UID that is assigned at the time of pre-
surveys
Total Sample# 20,240
Data Entry: Surveys took place in English, Kannada and Hindi. They were all
entered as is in Kannada and Hindi, then translated to English
Data Coding: Responses are both open and close-ended. Not all responses
receive points
37
Methodology: Focus Group Discussions (FGD)
Sample Selection: Two focus groups (12 participants/ group) were conducted for anhour, with students who were participants of the National Civic Fest, 2015
FGD Execution:
Two focus groups (12 participants/ group) were conducted for an hour, with studentswho were participants of the National Civic Fest, 2015.
The facilitators (Sayantani, Alia and Reema) guided the group through discussion andkept the group focused on the topic for discussion. The entire session was recordedand transcribed.
Focus Group Discussion Themes:
Nature of civic engagement before/after the Bala Janaagraha program;Conceptualizing the Ideal City, Role of Active Citizenship, Sustainable Development;Future engagement with the Bala Janaagraha program as Alumni.
Bala Janaagraha program-Themes, feedbacks, suggestions on The Resource Book, TheCivic Project, Post Program Engagement, etc.
38
Methodology: Bala Janaagraha Photography Project
Research Questions:
Does Bala Janaagraha shape how students perceive urban infrastructure challenges?
How do Bala Janaagraha students, as compared to non-Bala Janaagraha students, understand issues related to urban infrastructure?
Sample Selection: Five schools that had a Bala Janaagraha program and five schools that did not so that the
responses of the students.
All the students were 8th standard students.
Bala Janaagraha facilitators, who are tasked with running the program around Bangalore,
helped to identify schools that were and were not part of the Bala Janaagraha program
that would be willing to participate.
Schools included a mix of government and private schools with students from a variety of
socioeconomic backgrounds.
39
Methodology: Bala Janaagraha Photography Project
Data Collection:
Photo walk: Students then split into groups and walked around the area near their
school for 30 minutes to an hour
Each child was asked to fill out a worksheet in which they could write about up to
ten pictures
Students were asked to select pictures that they felt represented key positive and
negative issues for them and to write a few lines explaining what the picture showed
and why they had selected it
Student Interview on photography themes, city problems and solutions,
neighborhood
Data Cleaning and Analysis:
Once interviews and worksheets completed, respondent information was coded and
analyzed using the qualitative software, NVivo.
Responses were coded by themes and frequency: Garbage, Traffic, Environment, People,
Traffic, etc.
40
THANK YOU
41