ilm on wheels baseline assessment report - home | · pdf fileilm on wheels baseline assessment...

Funded by UKaid from the Department for International Development (DFID)

Ilm on Wheels

Baseline Assessment Report

October 2012

TeleTaleem (Pvt.) Limited

Plot 291, Street 3, I-9/3, Islamabad

Phone: +92 51 111 784 784 • Email: [email protected]

Website: www.teletaleem.com

Ilm on Wheels Project Baseline Assessment Report TeleTaleem

of 36

Table of Contents

1. Abbreviations and Acronyms .......................................................................................................................... 4

2. Background and objectives ............................................................................................................................ 5

2.1 Project background ................................................................................................................................ 5

2.2 Objectives of the baseline assessment .................................................................................................. 5

3. Methodology ................................................................................................................................................... 6

3.1 Identification of target area .................................................................................................................... 6

3.2 Identification and mapping of schools .................................................................................................... 7

3.3 Scoping survey ...................................................................................................................................... 7

3.3.1 Survey questionnaire ......................................................................................................................... 7

3.3.2 School survey and physical visits ...................................................................................................... 8

3.4 Sampling Design ................................................................................................................................... 8

3.5 School selection and treatment-control pairing ...................................................................................... 9

3.6 Pre-assessment ................................................................................................................................... 10

3.7 Satisfaction survey............................................................................................................................... 10

4. Findings and analysis ................................................................................................................................... 12

4.1 Baseline of Mathematics scores .......................................................................................................... 12

4.2 Baseline of student satisfaction ........................................................................................................... 13

4.2.1 Quality of Instructions ...................................................................................................................... 13

4.2.2 Quality of material, text book and teaching aids .............................................................................. 13

4.2.3 Duration of class and length of instructions ..................................................................................... 14

4.2.4 Opportunities for questions and answers, collective and individual counselling .............................. 15

5. Appendices................................................................................................................................................... 16

5.1 Scoping survey questionnaire – Part I ................................................................................................. 16

5.2 Scoping survey questionnaire – Part II ................................................................................................ 19

5.3 Satisfaction survey questionnaire ........................................................................................................ 22

5.4 Additional analysis ............................................................................................................................... 23

5.5 Score sheets of grade IV students ....................................................................................................... 28

5.6 Score sheets of grade V students ........................................................................................................ 32

List of Tables

Table ‎3.1: Comparison of education statistics across KPK ..................................................................................... 6

Table 3.2: Learning achievements in Maths and Science for Grade-IV students .................................................... 7

Table 3.3: Tentative list of target and control groups .............................................................................................. 9

Table 4.1: Distribution of average scores (%) and standard deviations of grade IV students ............................... 12

Table 4.2: Distribution of average scores (%) and standard deviations of grade V students ................................ 12

Table 4.3: Distribution of responses to questions 2, 3 and 9 ................................................................................. 13

Table 4.4: Distribution of responses to questions 6, 7, and 8 ................................................................................ 13

Table 4.5: Distribution of responses to questions 4, 5 and 12 ............................................................................... 14

Table 4.6: Distribution of responses to question 10 .............................................................................................. 14

Table 4.7: Distribution of responses to question 11 .............................................................................................. 14

Table 4.8: Distribution of responses to questions 13, 14 and 15 ........................................................................... 14


of 36

Table 4.9: Distribution of responses to questions 16, 17 and 18 ........................................................................... 15

Table 5.1: Average scores (%) of grade 4 students by union councils .................................................................. 23

Table 5.2: Average scores (%) of grade 5 students by union councils .................................................................. 23

Table 5.3: Average scores (%) of grade 4 students by sex of the students .......................................................... 24

Table 5.4: Average scores (%) of grade 5 students by sex of the students .......................................................... 24

Table 5.5: Average scores (%) of grade 4 students by teacher qualification ......................................................... 24

Table 5.6: Average scores (%) of grade 5 students by teacher qualification ......................................................... 25

Table 5.7: Average scores (%) of grade 4 students by years of experience ......................................................... 25

Table 5.8: Average scores (%) of grade 5 students by years of experience ......................................................... 25

Table 5.9: Average scores (%) of grade 4 students by sex of the teachers .......................................................... 26

Table 5.10: Average scores (%) of grade 5 students by sex of the teachers ........................................................ 26

Table 5.11: Average scores (%) of grade 4 students by class sizes ..................................................................... 26

Table 5.12: Average scores (%) of grade 5 students by class sizes ..................................................................... 26

Table 5.13: Average scores (%) of grade 4 students by availability of electricity .................................................. 27

Table 5.14: Average scores (%) of grade 5 students by availability of electricity .................................................. 27

Table 5.15: Average scores (%) of grade 4 students by school distance from main road .................................... 27

Table 5.16: Average scores (%) of grade 5 students by school distance from main road .................................... 27

Table 5.17: Score sheet of grade IV students ....................................................................................................... 28

Table 5.18: Score sheet of grade V students ........................................................................................................ 32


of 36

1. Abbreviations and Acronyms

ADO Assistant District Officer

DFID Department for International Development

DAI Development Alternatives, Inc.

EIF Education Innovation Fund

EMIS Education Management Information System

GPS Government Primary School

GGPS Government Girls Primary School

ICT Information and Communication Technology

KPK Khyber Pakhtunkhwa

NCHD National Commission for Human Development

NEAS National Education Assessment System

NEGD Non-Equivalent Group Design

UC Union Council


of 36

2. Background and objectives

2.1 Project background

Ilm Ideas is a three-year programme to award grants nationwide for the purpose of increasing access

to quality education for the children of Pakistan. Funded by UKaid from the Department for

International Development (DFID) and managed by Development Alternatives, Inc. (DAI) Europe

and Cambridge Education, Ilm Ideas seeks to support:

1. Research studies to inform policy and strategic interventions

2. Activities to increase demand for greater transparency and accountability

3. Creative solutions to the challenges faced by the education sector

The programme consists of two Funds, which award competitive grants in Pakistan based on strategic

priorities, geographic balance and established programming criteria. The Education Innovation Fund

(EIF) seeks to support bright ideas for tackling challenges faced by the education sector in Pakistan. It

aims to tie the energy of the active non-governmental and private sectors with the experience of the

governmental sector, to find solutions with the potential to be taken to scale.

TeleTaleem was granted funding for implementing a pilot project titled “Ilm on Wheels” under the

Education Innovation Fund to increase access to quality education and to enhance student satisfaction

at primary school level. The project runs for nine months (July 2012 to March 2013) covering a

cluster of public primary schools in the district of Mansehra.

The Ilm on Wheels project aims to leverage advance learning methods to attain broad based access

and improved quality of education in public primary schools. The project employs a mobile satellite-

enabled van to provide ICT enabled student learning and assessments, supported by an advanced

learning technology platform. The mobile van delivers completely self-contained services on a

transactional basis, anytime, anywhere, with no additional infrastructure requirements. The project

targets students of grades IV and V for enhanced learning achievements in Mathematics by

connecting them to the widely available but inaccessible online learning content and tools.

2.2 Objectives of the baseline assessment

The objective of the baseline assessment is to establish benchmarks for evaluating:

a) Improvement in learning achievements in Mathematics;

b) Increase in parent and/or student satisfaction as a result of this project.

The project baseline assessment serves a dual purpose; first to assess the students’ educational

attainment and help the project team effectively plan learning activities of the intervention; and

secondly to measure the change brought about by the intervention through a comparison of baseline

and end-of-project assessment values.


of 36

3. Methodology

The evaluation methodology is based on the Non-Equivalent Groups Design (NEGD), which is a

widely used method of quasi-experimental evaluations and social research. This design takes into

account before and after comparisons of project (treatment) and similar (but not identical) control

populations. This method of evaluation is usually used where randomization is not possible.

The NEGD is structured like a pre-test/post-test randomized experiment, but it lacks the key feature of

the randomized designs – random assignment. In the NEGD, intact groups are used that we think are

similar as the treatment and control groups. In education, we might pick two comparable classrooms

or schools. In community-based research, we might use two similar communities. We try to select

groups that are as similar as possible so we can fairly compare the treated one with the comparison

one. But we can never be sure the groups are comparable. Or, put another way, it's unlikely that the

two groups would be as similar as they would if we assigned them through a random lottery. Because

it's often likely that the groups are not equivalent, this design was named accordingly to highlight this

distinction.

Though with respect to internal validity, quasi-experimental designs often appear to be inferior to

randomized experiments, there is something compelling about these designs; they allow for some

level of reliable analysis of results. Such designs are also used due to budget, time and human

resource constraints – all of which are applicable in this pilot project.

3.1 Identification of target area

Mandated by the objectives and thematic areas set forth by the Education Innovation Fund, the search

focused on districts with low learning achievement, low transition and high drop-out rates. At the

provincial level, KPK was picked because of the low learning achievements in comparison to rest of

Pakistan. Within KPK, Mansehra was identified as the area of interest because all the aforementioned

education indicators were significantly lower in comparison to other districts of KPK (Table 3.1).

Table 33.1: Comparison of education statistics across KPK

No Indicator KPK Average Mansehra Districts worse off than Mansehra

1 Increase in Enrolment (2005-2010) 15.1% 3.8% Abbotabad, Karak, Tank, Hangu

2 Drop-out Rate 45% 59% Kohistan, Battagram

3 Transition Rate 77% 62% Kohistan, Battagram

(Source: Annual Statistical Report of Govt. Schools, 2010-11, KPK E&SE)

Several studies over the years have provided ample evidence of the quality of education in the rural

areas of Pakistan. 81% of the primary schools in Pakistan belong to rural areas with enrolments of

13.327 million students. 54% of these schools do not have electricity and the rest have to work

through extended periods of blackouts. Availability of computer equipment and internet, especially at

primary school level, is almost non-existent. For provinces like KPK and Balochistan, security risks

and gender related issues provide additional limitations. Student competency in subjects like Math

and Science, measured through standardized national assessments, also show weak learning

achievements for KPK and Balochistan.


of 36

Table 3.2: Learning achievements in Maths and Science for Grade-IV students

Province vs. Nation Mathematics Science

Rest of Pakistan 399.57 472.05

Khyber Pakhtunkhwa 370.65 438.91

Difference Statistically Significant

3.2 Identification and mapping of schools

The Education Management Information System (EMIS) data for elementary and secondary education

of Khyber Pakhtunkhwa was used in the identification of the primary schools in the tehsils of

Mansehra and Balakot. The basic criterion in the selection of the primary schools of the district was

the enrolments numbering more than 20 for girls and 30 for boys in the target grades (IV and V).

District Mansehra carries a mix of summer and winter zone schools, i.e. winter zone school holidays

are scheduled in winter season instead of the summer. Winter zone schools were not considered

because six months of schooling had already transpired in comparison to one month for the summer

zone schools. Hence the search within Mansehra zeroed in on Tehsils and Union Councils (UCs)

housing summer zone schools. Discussion with ADO Mansehra helped identify Tehsil Balakot as the

focal area, including 5 UCs with summer zone schools, i.e.:

1. Balakot

2. Garhi Habibullah

3. Talhatta

4. Karnol

5. Garlat

These UCs are co-located and have decent enough road network to support van access. The locations

of these schools were then mapped on cartographic software based on the clusters of rural and semi-

urban primary schools. Physical visits to the site and with the help of additional data provided by the

National Commission for Human Development (NCHD), a leading agency working on education in

the area, the list of potential schools was improved and several schools were either omitted or

included based on actual enrolment and ease of accessibility.

3.3 Scoping survey

A scoping survey of 48 schools in the district was carried out for collecting detailed information from

the schools. This scoping survey was designed to identify and collect information on the public

schools in the target area and to help in selection of the treatment and control schools.

Scoping survey was necessitated because of the gaps identified in available data, including EMIS. For

example, the data logged in EMIS was at least a year old and had numerous omissions in terms of

missing schools and incorrect entries. In addition some finer details only became apparent, once the

scoping survey was done. The scoping survey helped put a lot of corrective measures in the project

planning early on.

3.3.1 Survey questionnaire

A scoping survey questionnaire was designed to gather vital statistics of the schools identified for

scoping. The following variables were included in the scoping survey:


of 36

1. School name

2. Village name

3. Exact address

4. Accessibility status

5. Male / female students in grade IV

6. Male / female students in grade V

7. Status of classrooms as single or

multi-grade

8. Number of Math teachers

9. Details of qualification and

experience of Math teachers

10. Type of school building

11. Total number of classrooms in

school

12. Availability of electricity during

school hours (8am – 7pm)

13. Distance of school from main road

14. Total number of teachers in school

15. Total male and female students of

the school

16. Division of classes into sections

17. Last three year scores of grade IV

and V

The scoping survey contained questions designed for observation as well as eliciting specific

information about the schools. The questionnaire was divided into two parts:

a) Part I was filled by the school representative, preferably head teacher, which focused on

gathering information on the enrolment, class structures and sizes, and teaching staff, while

b) Part II of the survey contain questions related to interviewer observations on the physical

conditions and facilities that were recorded upon visits to the schools (appendices 5.1 and 5.2)

3.3.2 School survey and physical visits

Prior to the scoping survey, concerted efforts were made to ensure the attendance of representatives

from all the schools in the selected UCs, however only 65 heads of schools promised to arrive for the

survey. The data collection team trained on the questionnaire carried out the scoping survey in the

premises of the Assistant District Officer in Balakot where a total of 48 head teachers were present at

that time representing 48 schools out of the total 180 schools in the target UCs. Physical visits to 24 of

these schools for Part II of the survey resulted in previously unidentified schools which were listed

and scoped out to finalise the list of available schools.

The representatives of the schools were required to bring their enrolment sheets as well as results of

their respective classes for the past three years. Due to various factors, majority of the teachers could

not bring the required documents and were later contacted to ensure completeness of the information.

The data collected through the scoping survey was then digitized and processed to prepare the dataset

for final selection.

3.4 Sampling Design

The sampling design is based on the non-probability purposive sampling method. In accordance with

the project document, 10 treatment and 11 control classes/schools were selected based on pre-defined

criteria. The process of selection and criteria are presented in the following sections.


of 36

3.5 School selection and treatment-control pairing

Enrolment and accessibility were the primary criteria for eligibility of a school. 22 of the 48 schools

were removed from consideration due to low (actual) enrolment and inaccessibility. The remaining 26

schools were then segregated into the respective grades (IV and V) for both girls and boys primary

schools. In all, 9 schools for girls and 15 schools for boys were identified as potential candidates and

prepared for statistical analysis for pairing.

Class enrolment, teacher qualifications, years of experience in teaching, and the past grades in Maths

for the students of these schools were then analysed to determine the factors that would allow pairing

of similar classes between the treatment and control groups. It was imperative that the target and

control groups be similar in characteristics to establish the difference brought about in the learning

achievements of students in the target group compared to the control group. Past grades in Maths,

being a key decision criterion for pairing, could not be used definitively because it was based on non-

standardized tests. Therefore, results from the pre-assessment (covered in section 3.6) were analysed

in a subsequent iteration, to finally lock the control-treatment pairs of schools. Of the 26 schools, 21

classes from 13 schools were finalized for the intervention out of which three pairs of treatment-

control groups were identified for grade IV and three pairs for grade V. Table 3.3 presents the

tentative list of target and control groups, clearly matching those schools that have been paired.

Table 3.3: Tentative list of target and control groups

Grade Pair # Treatment Control

Class ID School Name Students Class ID School Name Students

4

1 451A GPS Jagir 33 455A GPS Talhatta 48

2 459A GPS Ghari Habib Ullah 42 459B GPS Ghari Habib Ullah 35

3 470A GGPS Hassari 20 466A GGPS Bagal Char 40

NP 408A GPS Shoal Najaf Khan 32

NP 423A GGPS Ghari Habibullah 30

NP

405A GPS Sugdhar 25

NP

421A GGPS Balakot 7

NP

441A GPS Banphora 26

Total 157 Total 181

5

4 551A GPS Jagir 30 541A GPS Banphora 37

5 559B GPS Ghari Habib Ullah 30 559A GPS Ghari Habib Ullah 35

6 570A GGPS Hassari 25 521A GGPS Balakot 11

NP 523A GGPS Ghari Habibullah 32

NP 555A GPS Talhatta 32

NP

511A GPS Zameeri 25

NP

571A GGPS Narrah 50

Total 154 Total 153

Note: NP = No Pairs

The final pairing of the treatment and control groups is being carried out with additional information

being collected on the schools in the five selected UCs.


of 36

3.6 Pre-assessment

In order to provide a baseline for assessing the impact of the intervention, a standardized pre-

assessment was designed and conducted in all the target and control groups. The pre-assessment test

was prepared with the help of the National Education Assessment System (NEAS) of the Ministry of

Education and Trainings, and was based on the assessment framework as defined in the national

curricula (2006) of Mathematics for grades IV and V. An elaborate Item testing activity was carried

out by running the pilot test over 687 students, from grade V and grade VI (for finalizing grade IV

and grade V assessment tools respectively). This pool of students was selected from within the

Balakot-Mansehra target area, staying clear of the designated control and treatment schools. The pre-

assessment was finalized on the basis of the responses to these questions filtering out the best fit

questions.

Next the pre-assessment of the treatment and control groups was carried out in two phases, one for

each grade, by a team of trained project staff. The students were given detailed briefings on the

questionnaire and procedures for answering the questions. For almost all of the children, the

standardized test was their first encounter with such testing methods and necessitated introduction to

the tool. After a half-hour tutorial, the students were given three hours to fill out their respective pre-

assessments. Each questionnaire was tagged with school ID and students were required to fill out their

names and roll numbers for identification.

In all, assessments of 496 students were carried out from an expected 576 students (i.e. total

enrolment in grade IV and V in both treatment and control groups). Out of the 576 students, 240

belonged to grade IV and 256 in grade V. While the pre-assessments were originally slated to be

conducted before the commencement of the intervention, the actual pre-assessment took place two

weeks in to the project launch. The effects of this delay have not been statistically analysed and

accounted for in the presentation of the findings in this report.

The results of the pre-assessment were graded on the basis of correct responses out of total questions

(60 and 58 for grades IV and V respectively) and their scores calculated as a percentage of correct

responses. The findings presented in this report are based on the average scores of the students.

3.7 Satisfaction survey

A pre-intervention student satisfaction survey was designed to be conducted alongside the pre-

assessment to serve as a benchmark for a post-intervention satisfaction study to be conducted at the

conclusion of the project. Based on published research, key questions were identified for the

satisfaction survey and were classified in the following four categories:

1. Quality of instructions

2. Quality of material, text book and teaching aids

3. Duration of class and length of instructions

4. Opportunities for questions and answers, collective and individual counseling

After review of multiple student satisfaction surveys, direct and indirect discussions and feedback

from the teachers and team members, 18 questions were devised to ensure appropriate coverage of the

four categories. A question specific Likert rating scale ranging from yes-no answers to four-point


of 36

“Very Satisfied” to “Very Dissatisfied” scales were used in the survey (see Appendix 5.3 –

Satisfaction Survey).

With the majority of the students facing difficulties in reading comprehension, the project staff

conducted the satisfaction surveys by guiding the students through each question and asking them to

mark their responses on paper.


of 36

4. Findings and analysis

4.1 Baseline of Mathematics scores

The results of the standardized pre-assessment conducted for grade IV and V students are presented in

Tables 4.1 and 4.2. The results are presented at class level and distributed across treatment and control

groups. On average grade IV students managed to score 22.89 out of 100 overall, while grade V

students scored 39.51 out of 100.

Grade IV students in the treatment group managed to obtain an overall average score of 24.87 out of

100 with a standard deviation of 12.57 (Table 4.1), while those in the control group scored an overall

average of 21.12 out of 100 with a standard deviation of 19.97.

The overall average score of grade V students in the treatment group was 39.88 out of 100 with a

standard deviation of 20.01 (Table 4.2) while those in the control group managed an average score of

39.14 out of 100 with a standard deviation of 17.86.

Table 4.1: Distribution of average scores (%) and standard deviations of grade IV students

Pair

Treatment Control

CID School Name and Class Average Scores

Standard Deviation


Standard Deviation

1 451A GPS Jagir 6.50 4.97 455A GPS Talhatta 9.71 7.15

2 459A GPS Ghari Habibullah - A 31.31 9.16 459B GPS Ghari Habibullah - B 24.47 7.17

3 470A GGPS Hassari 22.60 6.95 466A GGPS Bagal Char 19.81 5.35

408A GPS Shoal Najaf Khan 36.67 6.96

423A GGPS Ghari Habibullah 21.88 7.90

405A GPS Sugdhar 10.60 8.08

421A GGPS Balakot 83.33 4.44

441A GPS Banphora 14.09 8.28

Overall 24.87 12.57 Overall 21.12 19.97

Table 4.2: Distribution of average scores (%) and standard deviations of grade V students

Pair

Treatment Control


Standard Deviation


Standard Deviation

1 551A GPS Jagir 30.44 11.30 541A GPS Banphora 34.77 6.15

2 559B GPS Ghari Habibullah - B 22.17 5.60 559A GPS Ghari Habibullah - A 33.07 14.40

3 570A GGPS Hassari 26.23 10.12 521A GGPS Balakot 73.04 4.84

523A GGPS Ghari Habibullah 70.69 6.20

555A GPS Talhatta 42.27 9.97

511A GPS Zameeri 21.59 10.22

571A GGPS Narrah 47.39 12.88

Overall 39.88 20.01 Overall 39.14 17.86


of 36

4.2 Baseline of student satisfaction

The responses received from the students on the satisfaction survey are presented below in their

respective categories. On average about 16% of the students chose not to respond to the questions,

while around 2.6% made mistakes (incorrect selection) while recording their responses. The

percentages detailed in the analysis below does not account for these mistakes.

4.2.1 Quality of Instructions

Seven of the questions posed in the satisfaction survey were based on the quality of instruction in the

class rooms. The overall satisfaction levels for quality of instructions are mostly positive with more

than 70% of the students responding positively to these questions (Table 4.3). 13.7% of the students

do not always understand what is being taught in the classes.

Table 4.3: Distribution of responses to questions 2, 3 and 9

Q# Question Yes Sometimes No No Response

2 Do you understand what you are taught in math class? 73.42 13.70 1.64 9.61

3 Do you enjoy math class? 78.32 4.29 2.66 12.47

9 Do you enjoy math exercises and quiz? 72.19 5.73 1.64 17.79

A little over half of the students believe that they receive class and home works, and maths tests on a

regular basis. On average, 14% of the students believe that the class and home activities happen

sometimes or even rarely. Quizzes and tests in maths account for the larger shares of activities that do

not happen regularly (Table 4.4).

Table 4.4: Distribution of responses to questions 6, 7, and 8

Q# Question Regularly Sometimes Rarely Never No

Response

6 Do you get math classwork for solving exercise questions? 57.46 13.50 11.04 2.25 15.75

7 Do you get math homework? 58.69 11.25 13.09 1.43 15.54

8 Do you get math test or quiz? 43.97 17.18 19.63 0.61 18.61

4.2.2 Quality of material, text book and teaching aids

Around 66% of the children are able to comprehend the examples and exercises presented in the text

book for mathematics. 13% of the students do not always understand the examples given in the text

books while 11% have trouble understanding the exercises (Table 4.5).


of 36


Q# Question Yes Sometimes No No

Response

4 Do you understand the examples of the math text book? 66.26 12.68 3.68 14.72

5 Do you understand the exercises of the math text book? 65.85 11.25 2.86 16.56

12 Do you enjoy using teaching material other than math text book? 73.21 6.34 1.64 15.95

66% of the students identified “black boards” as the most used teaching material used in their classes.

The use of flash cards (6.54%), computers (6.75%) or even blocks (2.25%) is limited to a very small

set of students.

Table 4.6: Distribution of responses to question 10

Q# Question Black board

Flash cards

Computer Blocks No

Response

10 Which of the following teaching material do you use for learning math?

66.46 6.54 6.75 2.25 18.00

45% of the students reported that the use of teaching material other than their textbook was happening

regularly. 25% of the students believe it to happen sometimes while 9.41% students reported rare uses

of any material other than their textbooks (Table 4.7).

Table 4.7: Distribution of responses to question 11

Q# Question Regularly Sometimes Rarely Never No Response

11 How often do you use teaching material other than the textbook?

45.60 25.56 9.41 0.61 18.81

4.2.3 Duration of class and length of instructions

For 66% of the students, there is enough time spent in class on learning, asking questions and doing

exercises. Almost 10% of the students do not always get enough time while another 3.9% students

never have enough time to spend on activities in class.


Q# Question Yes Sometimes No No

Response

13 Do you get enough time for learning math lessons in class? 65.44 9.20 3.89 18.20

14 Do you get enough time for asking question answers in class? 66.67 8.79 3.68 17.38

15 Do you get enough time for math exercises in class? 65.44 10.02 4.09 17.59


of 36

4.2.4 Opportunities for questions and answers, collective and individual

counselling

46% of the students believe that they rarely face any difficulties in solving math problems, while

18.4% of the students face difficulties on a regular basis. 14.72% of the students rarely get individual

counselling while 18.61% of the students are only sometimes able to solicit feedback on their class

and homework activities.


Q# Question Regularly Sometimes Rarely Never No Response

16 How often do you face difficulties in solving math problems?

18.40 13.70 45.81 5.32 16.77

17 How often are you individually able to get teacher counselling on math?

56.03 10.43 14.72 1.23 17.59

18 How often are you able to get feedback on the math classwork and homework?

56.44 18.61 7.98 1.23 15.75

Student satisfactions were conducted for each class and the findings shown above have highlighted

the tendency for bias based on several influencing factors, i.e. cheating, copying, teacher

interventions, etc. In order to prepare a more effective baseline for student satisfaction, further in

depth qualitative surveys will be conducted with a purposively selected group of students. These

surveys will be conducted in the coming months in controlled environments to ensure a more

comprehensive student satisfaction baseline.


of 36

5. Appendices

5.1 Scoping survey questionnaire – Part I


of 36


of 36

5.2 Scoping survey questionnaire – Part II


of 36


of 36

5.3 Satisfaction survey questionnaire

Q # Question Rating scale

1 Do you have regular math classes? Yes No

2 Do you understand what you are taught in math class?

Yes-Very much Yes-Somewhat No

3 Do you enjoy math class? Yes-Very much Yes-Somewhat No

4 Do you understand the examples of the math text book?


5 Do you understand the exercises of the math text book?


6 Do you get math classwork for solving exercise questions?

Yes-Regularly Yes-Sometimes Yes-Rarely No-Never

7 Do you get math homework? Yes-Regularly Yes-Sometimes Yes-Rarely No-Never

8 Do you get math test or quiz? Yes-Regularly Yes-Sometimes Yes-Rarely No-Never

9 Do you enjoy math exercises and quiz?


10 Which of the following teaching material do you use for learning math?

Black board Flash cards Computer Blocks

11 How often do you use teaching material other than the text book?

Regularly Sometimes Rarely Never

12 Do you enjoy using teaching material other than math text book?


13 Do you get enough time for learning math lessons in class?


14 Do you get enough time for asking question answers in class?


15 Do you get enough time for math exercises in class?


16 How often do you face difficulties in solving math problems?


18 How often are you individually able to get teacher counselling on math?


19 How often are you able to get feedback on the math classwork and homework?



of 36

5.4 Additional analysis

In the following sections, additional analysis is presented relating to student scores. Several variables

on which information is available were compared to student scores as an attempt to highlight other

factors that may influence student results and learning achievements. The reader should note that the

analysis presented is not conclusive of any relationship between student scores and related factors

since the research design does not support drawing such conclusions. However, this analysis does

present the opportunity to explore various factors through future research.

Furthermore, it should be noted that the analysis presented in these sections is not tied to the treatment

control pairs, but is rather presented as a broader comparison between students of treatment and

control schools.

Comparison of scores by Union Council

Table 5.1: Average scores (%) of grade 4 students by union councils

Union Councils

Treatment Control Overall

N Column

N % Avg.

Scores N

Column N %

Avg. Scores

N Column

N % Avg.

Scores

Balakot 22 19.30 36.67 31 24.41 34.19 53 21.99 35.22

Gari Habibullah 72 63.16 26.37 42 33.07 25.83 114 47.30 26.17

Garlat 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00

Karnol 20 17.54 6.50 0 0.00 0.00 20 8.30 6.50

Talhatta 0 0.00 0.00 54 42.52 9.94 54 22.41 9.94

Total 114 100.00 24.87 127 100.00 21.12 241 100.00 22.89

Table 5.2: Average scores (%) of grade 5 students by union councils

Union Councils


N Column

N % Avg.

Scores N

Column N %

Avg. Scores

N Column

N % Avg.

Scores

Balakot 0 0.00 0.00 35 28.46 46.80 35 14.11 46.80

Gari Habibullah 70 56.00 42.39 28 22.76 33.07 98 39.52 39.73

Garlat 0 0.00 0.00 37 30.08 47.39 37 14.92 47.39

Karnol 26 20.80 30.44 0 0.00 0.00 26 10.48 30.44

Talhatta 29 23.20 42.27 23 18.70 21.59 52 20.97 33.12

Total 125 100.00 39.88 123 100.00 39.14 248 100.00 39.51


of 36

Scores disaggregated by sex

Table 5.3: Average scores (%) of grade 4 students by sex of the students

Sex


N Column

N % Avg.

Scores N

Column N %

Avg. Scores

N Column

N % Avg.

Scores

Female 56 49.12 19.94 52 40.94 22.60 108 44.81 21.22

Male 58 50.88 29.63 75 59.06 20.09 133 55.19 24.25

Total 114 100.00 24.87 127 100.00 21.12 241 100.00 22.89

Table 5.4: Average scores (%) of grade 5 students by sex of the students

Sex


N Column

N % Avg. Scores N

Column N %

Avg. Scores

N Column

N % Avg.

Scores

Female 60 48.00 49.97 68 55.28 44.04 128 51.61 46.82

Male 65 52.00 30.56 55 44.72 33.07 120 48.39 31.71

Total 125 100.00 39.88 123 100.00 39.14 248 100.00 39.51

Student scores in comparison to teacher qualification

Table 5.5: Average scores (%) of grade 4 students by teacher qualification

Teacher Qualification


N Column

N % Avg.

Scores N

Column N %

Avg. Scores

N Column

N % Avg.

Scores

Matric 22 19.30 36.67 14 11.02 10.60 36 14.94 26.53

FA 53 46.49 21.95 31 24.41 34.19 84 34.85 26.47

BA 23 20.18 21.88 9 7.09 19.81 32 13.28 21.30

BSc 0 0.00 0.00 40 31.50 9.71 40 16.60 9.71

B.Ed 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00

M.Ed 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00

Unknown 16 14.04 22.60 33 25.98 27.47 49 20.33 25.88

Total 114 100.00 24.87 127 100.00 21.12 241 100.00 22.89


of 36

Table 5.6: Average scores (%) of grade 5 students by teacher qualification

Teacher Qualification


N Column

N % Avg.

Scores N

Column N %

Avg. Scores

N Column

N % Avg.

Scores

Matric 28 22.40 70.69 0 0.00 0.00 28 11.29 70.69

FA 29 23.20 42.27 35 28.46 46.80 64 25.81 44.75

BA 26 20.80 30.44 0 0.00 0.00 26 10.48 30.44

BSc 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00

B.Ed 0 0.00 0.00 23 18.70 21.59 23 9.27 21.59

M.Ed 0 0.00 0.00 28 22.76 33.07 28 11.29 33.07

Unknown 42 33.60 23.52 37 30.08 47.39 79 31.85 34.70

Total 125 100.00 39.88 123 100.00 39.14 248 100.00 39.51

Relationship of scores to years of teaching experience

Table 5.7: Average scores (%) of grade 4 students by years of experience

Years of Experience


N Column

N % Avg.

Scores N

Column N %

Avg. Scores

N Column

N % Avg.

Scores

Less than 10 years 0 0.00 0.00 40 31.50 9.71 40 16.60 9.71

Between 10 and 20 years 45 39.47 29.11 23 18.11 14.20 68 28.22 24.07


More than 30 years 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00

Unknown 16 14.04 22.60 33 25.98 27.47 49 20.33 25.88

Total 114 100.00 24.87 127 100.00 21.12 241 100.00 22.89

Table 5.8: Average scores (%) of grade 5 students by years of experience

Years of Experience


N Column

N % Avg.

Scores N

Column N %

Avg. Scores

N Column

N % Avg.

Scores

Less than 10 years 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00



More than 30 years 0 0.00 0.00 24 19.51 34.77 24 9.68 34.77

Unknown 42 33.60 23.52 37 30.08 47.39 79 31.85 34.70

Total 125 100.00 39.88 123 100.00 39.14 248 100.00 39.51


of 36

Relationship of scores to sex of the teachers

Table 5.9: Average scores (%) of grade 4 students by sex of the teachers

Sex of the teachers


N Column

N % Avg.

Scores N

Column N %

Avg. Scores

N Column

N % Avg.

Scores

Female 39 34.21 22.18 18 14.17 51.57 57 23.65 31.46

Male 75 65.79 26.27 109 85.83 16.09 184 76.35 20.24

Total 114 100.00 24.87 127 100.00 21.12 241 100.00 22.89

Table 5.10: Average scores (%) of grade 5 students by sex of the teachers

Sex of the teachers


N Column

N % Avg.

Scores N

Column N %

Avg. Scores

N Column

N % Avg.

Scores

Female 42 33.60 55.87 48 39.02 53.27 90 36.29 54.48

Male 83 66.40 31.78 75 60.98 30.09 158 63.71 30.98

Total 125 100.00 39.88 123 100.00 39.14 248 100.00 39.51

Scores with comparison to class size

Table 5.11: Average scores (%) of grade 4 students by class sizes

Class Sizes


N Column

N % Avg.

Scores N

Column N %

Avg. Scores

N Column

N % Avg.

Scores

Less than 10 students 0 0.00 0.00 18 14.17 51.57 18 7.47 51.57

Between 10 and 20 students 16 14.04 22.60 14 11.02 10.60 30 12.45 17.00


More than 30 students 33 28.95 31.31 73 57.48 17.74 106 43.98 21.97

Total 114 100.00 24.87 127 100.00 21.12 241 100.00 22.89

Table 5.12: Average scores (%) of grade 5 students by class sizes

Class Sizes


N Column

N % Avg.

Scores N

Column N %

Avg. Scores

N Column

N % Avg.

Scores

Less than 10 students 0 0.00 0.00 0 0.00 0.00 0 0.00 0.00



More than 30 students 0 0.00 0.00 37 30.08 47.39 37 14.92 47.39

Total 125 100.00 39.88 123 100.00 39.14 248 100.00 39.51


of 36

Availability of electricity

Table 5.13: Average scores (%) of grade 4 students by availability of electricity

Availability of Electricity


N Column

N % Avg.

Scores N

Column N %

Avg. Scores

N Column

N % Avg.

Scores

Yes 78 68.42 30.04 40 31.50 9.71 118 48.96 23.15

No 36 31.58 13.66 87 68.50 26.36 123 51.04 22.64

Total 114 100.00 24.87 127 100.00 21.12 241 100.00 22.89

Table 5.14: Average scores (%) of grade 5 students by availability of electricity

Availability of Electricity


N Column

N % Avg.

Scores N

Column N %

Avg. Scores

N Column

N % Avg.

Scores

Yes 57 45.60 56.23 28 22.76 33.07 85 34.27 48.60

No 68 54.40 26.17 95 77.24 40.93 163 65.73 34.77

Total 125 100.00 39.88 123 100.00 39.14 248 100.00 39.51

Distance from school

Table 5.15: Average scores (%) of grade 4 students

by school distance from main road

Distance from main road


N Column

N % Avg.

Scores N

Column N %

Avg. Scores

N Column

N % Avg.

Scores

Less than 1 KM 75 65.79 26.27 22 17.32 14.09 97 40.25 23.51

Less than 2 KM 23 20.18 21.88 23 18.11 39.06 46 19.09 30.47

Less than 5 KM 0 0.00 0.00 49 38.58 11.56 49 20.33 11.56

More than 5 KM 16 14.04 22.60 33 25.98 27.47 49 20.33 25.88

Total 114 100.00 24.87 127 100.00 21.12 241 100.00 22.89

Table 5.16: Average scores (%) of grade 5 students

by school distance from main road

Distance from main road


N Column

N % Avg.

Scores N

Column N %

Avg. Scores

N Column

N % Avg.

Scores

Less than 1 KM 26 20.80 30.44 52 42.28 33.85 78 31.45 32.71

Less than 2 KM 28 22.40 70.69 34 27.64 38.24 62 25.00 52.89

Less than 5 KM 29 23.20 42.27 0 0.00 0.00 29 11.69 42.27

More than 5 KM 42 33.60 23.52 37 30.08 47.39 79 31.85 34.70

Total 125 100.00 39.88 123 100.00 39.14 248 100.00 39.51


of 36

5.5 Score sheets of grade IV students

Table 5.17: Score sheet of grade IV students

Grade SID Section Roll No. Sex School UC Status Score

4 5 A 1 M GPS Sugdhar Talhatta Control 25.0











4 5 A 17 F GPS Sugdhar Talhatta Control 13.3



4 8 A 2 M GPS Shoal Najaf Khan Balakot Treatment 40.0














4 8 A 17 F GPS Shoal Najaf Khan Balakot Treatment 38.3








4 21 A 1 F GGPS Balakot Balakot Control 73.3



4 21 A 4 M GGPS Balakot Balakot Control 86.7






4 23 A 1 F GGPS Ghari Habibullah Gari Habibullah Treatment 15.0






4 23 A 7 M GGPS Ghari Habibullah Gari Habibullah Treatment 35.0


of 36


















4 41 A 1 M GPS Banphora Balakot Control 26.7







4 41 A 9 F GPS Banphora Balakot Control 3.3















4 51 A 1 M GPS Jagir Karnol Treatment 8.3









4 51 A 13 F GPS Jagir Karnol Treatment 13.3










of 36




4 55 A 1 M GPS Talhatta Talhatta Control 33.3




4 55 A 5 F GPS Talhatta Talhatta Control 23.3




































4 59 A 1 M GPS Ghari Habib Ullah Gari Habibullah Treatment 31.7















of 36

















4 59 A 36 F GPS Ghari Habib Ullah Gari Habibullah Treatment 16.7




4 59 B 1 M GPS Ghari Habib Ullah Gari Habibullah Control 35.0



4 59 B 6 F GPS Ghari Habib Ullah Gari Habibullah Control 8.3






























4 66 A 1 F GGPS Bagal Char Gari Habibullah Control 18.3





of 36







4 70 A 1 F GGPS Hassari Gari Habibullah Treatment 28.3

4 70 A 2 M GGPS Hassari Gari Habibullah Treatment 11.7















5.6 Score sheets of grade V students

Table 5.18: Score sheet of grade V students

Grade SID Section Roll No. Sex School UC Status Score

5 11 A 1 M GPS Zameeri Talhatta Control 1.7













5 11 A 16 F GPS Zameeri Talhatta Control 1.7















of 36



























































of 36































5 55 A 1 M GPS Talhatta Talhatta Treatment 34.5
















5 55 A 19 F GPS Talhatta Talhatta Treatment 48.3












of 36




5 59 A 1 M GPS Ghari Habib Ullah Gari Habibullah Control 48.3























5 59 A 28 F GPS Ghari Habib Ullah Gari Habibullah Control 24.1





5 59 B 1 M GPS Ghari Habib Ullah Gari Habibullah Treatment 20.7













5 59 B 16 F GPS Ghari Habib Ullah Gari Habibullah Treatment 25.9














of 36


















5 71 A 2 F GGPS Narrah Garlat Control 70.7





































ilm on wheels baseline assessment report - home | · pdf fileilm on wheels baseline assessment...

Documents