January, 2018

Working Paper No. 1022

Data and Policy Decisions: Experimental

Evidence from Pakistan

Michael Callen

Saad Gulzar

Ali Hasanain

Muhammad Yasir Khan

Arman Rezaee

Data and Policy Decisions: Experimental Evidence

from Pakistan∗

Michael Callen†

Saad Gulzar‡

Ali Hasanain§

Muhammad Yasir Khan¶

Arman Rezaee‖

January 2018

Abstract

We evaluate a program in Pakistan that equips government health inspectors with asmartphone app which channels data on rural clinics to senior policy makers. Data aretransmitted in real-time, and submitted reports require timestamps, geostamps, andphotos. The system led to rural clinics being inspected 74% more often. In addition,we test whether senior officials act on the information provided by the system. Wefind that highlighting poorly performing facilities raises doctor attendance by 18.1percentage points from a dismal base of 24.5%. Our results indicate that technologycan mobilize data to real effect, even in low capacity settings.

∗Authors’ Note: This paper combines two previous papers circulated with the titles “The Political Economy of PublicSector Absence: Experimental Evidence from Pakistan” and “Personalities and Public Sector Absence: Evidence from aHealth Experiment in Pakistan”. We thank Farasat Iqbal for championing and implementing the project and Asim Fayazand Zubair Bhatti for designing the smartphone monitoring program. Support is generously provided by the InternationalGrowth Centre (IGC) political economy program, the IGC Pakistan Country Office, and the University of California Office ofthe President Lab Fees Research Program Grant #235855. Callen was supported by grant #FA9550-09-1-0314 from the AirForce Office of Scientific Research. We thank Erlend Berg, Eli Berman, Leonardo Bursztyn, Ali Cheema, Melissa Dell, RubenEnikolopov, Barbara Geddes, Naved Hamid, Gordon Hanson, Michael Kremer, Asim Ijaz Khwaja, Craig McIntosh, Ijaz Nabi,Aprajit Mahajan, Monica Martinez-Bravo, Benjamin A. Olken, Gerard Padro-i-Miquel, Karthik Muralidharan, Rohini Pande,Daniel N. Posner, Ronald Rogowski, Jacob N. Shapiro, Christopher Woodruff, Oliver Vanden Eynde, David Yanagizawa-Drott,Ekaterina Zhuravskaya and various seminar participants for insightful comments. Excellent research assistance was provided byMuhammad Zia Mehmood and Haseeb Ali. We thank Ali Cheema and Farooq Naseer for kindly sharing their data on electionoutcomes.†University of California, San Diego and NBER. mjcallen@ucsd.edu‡Stanford University. gulzar@stanford.edu.§Lahore University of Management Sciences. hasanain@lums.edu.pk¶University of California, Berkeley. yasir.khan@berkeley.edu‖University of California, Davis. abrezaee@ucdavis.edu

1 Introduction

Information technology is providing governments across the globe with greater access to data

to inform policymaking. Technologies like smartphones and tablets make it simple and cheap

to collect, compile, and visualize specialized and timely information relevant to a range of

decisions. Increasingly, policymakers no longer need to rely on the collection and aggregation

of geographically disparate paper records to understand how their government is operating.

They can have this information instantly, and presented in the form that best suits their

needs.

But will policymakers use these data? Will activating such technologies improve the qual-

ity of service provision? These are complex questions where the capabilities of government

personnel, the specific government organization, and the broader political and institutional

environment all potentially impact the answers.

To provide evidence on these questions, we conduct a randomized controlled evaluation

of a smartphone monitoring program in Punjab, Pakistan. The program—officially termed

‘Monitoring the Monitors’—equips government inspectors with a smartphone application

that collects data and feeds it to an online dashboard system. This provides real-time

information on rural public health clinics in Punjab, Pakistan, aggregated into simple charts

and tables for the review of senior health officials.1 It also includes several fail-safes to ensure

accurate reporting: reports are geo-stamped and time-stamped and all staff reported present

must be photographed with the inspector. In this environment, irregular inspections (in our

baseline, only 23% of facilities had received their required monthly inspection) and doctor

absence (doctors were present at 22% of facilities during regular operating hours in our

baseline) are serious issues. The smartphone system supplanted the previous paper-based

system for collecting operational data on public health facilities, which rarely functioned.

The evaluation spans 35 of 36 districts in Punjab.2 Punjab is a province of 100 mil-

1The data include staff attendance, availability of medicine, patient visits, vaccines provided, cleanliness,and so on.

2One district was withheld to pilot the system.

1

lion people, with many citizens utilizing public health services. The experiment involved

117 inspectors, 35 senior officers, 2,496 rural health clinics (of which, we sample 850), and

took place across 240 different parliamentary assembly constituencies. We therefore evaluate

the system at scale.3 This setting also provides variation to begin to speculatively exam-

ine whether individual and institutional constraints are relevant to the performance of the

system.

Beyond improving the flow of data, the system also changed the behavior of inspectors.

Using data from three rounds of independent audits in rural health facilities over the course

of a year, we find that the program increased the inspection rate from 24.5% in control

districts to 42.6% in treatment districts.4

We also built into the experiment a feature that allows us to examine whether providing

senior officials with data changes their behavior. Specifically, if more than three of the seven

health workers that are supposed to staff a rural clinic are absent during a health inspection,

we ‘flagged’ a facility as underperforming by highlighting it in red on the dashboard.5 We

test for effects on officials’ behavior by examining whether doctor attendance increases in

flagged facilities. We find that flagging increases doctor attendance from 23.6% to 41.3%

(standard error of difference = 8.2 percentage points), while it has no effect on the attendance

of other, less senior, clinic staff. We interpret this as evidence that policymakers use data

when making decisions.

Our study principally concerns the potential for information technology and data to

increase accountability and improve policy (Blum and Pande, 2015; Callen and Long, 2015;

Callen et al., 2016; Nealer et al., 2017), but relates to three additional literatures. The

first regards whether monitoring of government workers improves attendance (Banerjee and

3Indeed, at the conclusion of the evaluation, the system was scaled to cover the province and continuesto operate.

4The standard error of the difference clustered at the district level is 6.6 percentage points. With 35clusters, asymptotic reference distributions may not be valid, so we also use Fisher exact tests which do notappeal to asymptotic distributions. The fisher exact p-value of the difference is 0.001.

5We selected the threshold of three as this gave us the largest mass of inspections and so afforded themost statistical power.

2

Duflo, 2006; Banerjee et al., 2008; Chaudhury et al., 2006; Dhaliwal and Hanna, 2017).

The second broadly studies incentives in bureaucracies in developing countries (Ashraf et

al., 2014, 2015; Bertrand et al., 2017; Dal Bo et al., 2013; Deserranno, 2017; Finan et al.,

2017; Gulzar and Pasquale, 2017; Khan et al., 2016; Rogger and Rasul, forthcoming; Xu,

2017). The final involves experiments at scale (Muralidharan et al., 2016; Muralidharan and

Niehaus, 2017).

The paper proceeds as follows. Section 2 provides background on the ‘Monitoring the

Monitors’ program. Section 3 describes the data and the experiment. Section 4 provides

results, and section 5 concludes.

2 Background

2.1 Public Health Services in Punjab

In Punjab, public health services are provided by the Department of Health, headquartered

in Lahore and headed by the Secretary of Health. The provincial Department of Health

comprises 36 District Health Departments, each headed by an Executive District Officer,

hereafter referred to as ‘senior health officials’, who report directly to the Secretary. It

is straightforward for the Secretary to hold senior health officials accountable for service

delivery. Performance by senior health officials is also commonly rewarded with appointment

to a higher office. These officials each oversee public health services affecting several million

people. In this paper, we explore whether they, specifically, respond to performance data

from the facilities they oversee.

Senior health officials are each supported by several Deputy District Officers, hereafter

referred to as ‘health inspectors’, typically one for each sub-district (there are, on average,

3.4 sub-districts per district). Health inspectors are charged with inspecting all of the health

facilities in their sub-district at least once every month. Indeed, they are almost exclusively

tasked with inspections. There are five classifications of health facilities; we focus on the

3

frontline tier, called Basic Health Units, hereafter referred to as ‘health clinics’. There are

2,496 such health clinics in Punjab, almost all of which operate in rural and peri-urban areas.6

Health clinics typically consist of a only a few rooms. They provide several basic services,

including outpatient treatments, neonatal and reproductive healthcare, and vaccinations.

Each health clinic is headed by a doctor, known as the Medical Officer, who is supported

by a Dispenser, a Lady Health Worker, a Health Technician, a School Health and Nutrition

Supervisor, a Computer Operator, and a Midwife. Officially, health clinics are open from

8am to 2pm, Monday through Saturday.7

Unlike their superiors, health inspectors rarely ascend to higher leadership positions.

And, while they are neither officially or practically accountable for service delivery in the

same way that senior health officials are, they do have the authority to punish absent clinic

staff by issuing a ‘show-cause notice’, which requires staff to explain their absence to senior

health officials. They can also suspend and deny pay to contract staff, including doctors.

In severe cases of persistent absence, staff can also be transferred to less desirable locations,

but almost never fired.

Medical Officers are of particular interest for this study. These doctors are general prac-

titioners who have completed five years of medical school, and are therefore the most trained

health professionals in rural areas. Doctors are either hired centrally as permanent employ-

ees of Punjab’s Department of Health, or they are hired at the district level on a contractual

basis by a senior bureaucrat. While doctors receive higher income with rising seniority, their

portfolio of duties does not usually increase significantly. Very few doctors rise through the

ranks to become health inspectors: compared to the 2,496 Medical Officer posts in clinics,

there are only 123 such senior positions. Figure 1 depicts this simplified health administra-

tion hierarchy in Punjab.

6Each health clinic serves one Union Council, which is the smallest administrative unit in Pakistan.7Medical Officers, Dispensers, Computer Operators, and Health Technicians are all supposed to be at

the clinic during operating hours, while Midwives, Lady Health Workers, and School Health and NutritionSpecialists are typically traveling to provide services. Some clinics also have additional Midwives and LadyHealth Workers.

4

Figure 1: Organization Chart for the Punjab Department of Health

2.2 Pre-existing Paper-Based Monitoring System

During their required monthly inspections, health inspectors are required to collect infor-

mation on a standard paper form. This form records utilization, resource availability, and

worker absence. We provide this form in Appendix B.8 Once collected, forms are brought to

a central district facility, manually entered into a spreadsheet, and aggregated into a monthly

report for senior health officials.

This inspection system affords only limited visibility into inspectors’ activities to senior

officials. Compounding this problem, senior health officials have only two weak means of

sanctioning an inspector: issuing a verbal reprimand or, in serious cases, sending a written

request for investigation to provincial authorities. The investigation process is long, highly

bureaucratic, and, anecdotally, prone to interference by elected politicians.

2.3 ‘Monitoring the Monitors’ Smartphone Monitoring Program

We partnered with the Department of Health to design and experimentally evaluate the

‘Monitoring the Monitors’ program. This program replaced the existing paper-based moni-

8Appendix A reports ancillary results.

5

toring system with an Android-based smartphone application, which collects the same data

as the paper forms and transmits them instantly to a central online dashboard for the Sec-

retary of Health and senior health officials.9 The dashboard provides summary statistics,

charts, and graphs in a format designed in collaboration with senior health officials. Inspec-

tions are geotagged, timestamped, and require photos of the inspector and all health clinic

staff marked present to check for reliability. The geotagging and timestamping features are

designed to ensure inspectors visit health clinics while the staff photos are intended to ensure

that the digital reports of staff attendance are accurate.

Figure 2, Panel A, depicts the view of the dashboard that the Secretary of Health sees

when first logging on. It presents a bar chart giving the number of health clinic inspections

conducted in a district as a proportion of number of inspections assigned that month, allowing

the Secretary to compare performance across districts. Panel B provides an alternate view

available to senior health officials—a summary spreadsheet where each row corresponds to

a different health clinic inspection that occurred a senior health official’s district.

In Section 3.2, we provide full details of our experimental evaluation of the ‘Monitoring the

Monitors’ program. Our design allows us to estimate the effect of the reform on health clinic

inspections and on doctor attendance, and, separately, the effect of providing information to

senior health officials via the dashboard.

3 Data and Experiment

3.1 Data

To measure the impacts of our smartphone monitoring on health clinic inspections and doctor

attendance, we collected primary data on a representative sample of 850 (34%) of the 2,496

9Appendix C provides the training manual for the mobile application provided to inspectors and Ap-pendix D provides the training manual provided to senior health officials to assist them in using the dash-board.

6

Panel A: Summary of Health Clinic Inspection Compliance by District

Panel B: Summary of Health Clinic Inspections within a District

Figure 2: Online Dashboard Screenshots

7

ControlTreatmentPilot District - Khanewal

Figure 3: Locations of Health Clinics Surveyed

Notes: Borders demarcate districts in Punjab.

health clinics in Punjab.10 All districts in Punjab except Khanewal are represented in our

data.11 To our knowledge, this is the first representative survey of health clinics in Punjab.

Figure 3 provides a map of the health clinics in our experimental sample along with district

boundaries.

Enumerators made three unannounced visits to these 850 health clinics: one before smart-

phone monitoring began, in November 2011, and two after smartphone monitoring began

(in treatment districts), in June and October 2012.12 Surveys took three weeks to field in

each wave.

During these unannounced visits, enumerators collected the same information that health

inspectors record—information on health clinic utilization, resource availability, and worker

absence—as well as information on the occurrence of health clinic inspections themselves.

10Health clinics were selected randomly using an Equal Probability of Selection (EPS) design, stratifiedon district and distance from the district headquarters. Our estimates of performance are thus self-weighting,and no sampling corrections are used in the analysis.

11The smartphone technology was piloted in Khanewal.12Our survey teams were trained by senior enumerators and our team members at four regional hubs.

Following these trainings, the teams made visits to health clinics in their assigned districts and remained inregular contact with their team leaders and our research team.

8

Enumerators physically verified health clinic staff presence, filling out an attendance sheet

at the end of their visit and in private for doctors as well as dispensers, lady health work-

ers, health/medical technicians, school health and nutrition supervisors, and midwives.13

Health inspectors record visits by signing paper registers maintained at the health facility.

Enumerators measured whether a health inspection occurred in the prior month by inter-

viewing facility staff and verifying the register record.14 Data collection and entry followed

back-checks and other validation processes consistent with academic best practice. Summary

statistics from unannounced audits at baseline are presented in in Appendix Table A1.

We also conducted face-to-face time use surveys with all health inspectors in Punjab

between February and March 2013. During the survey, inspectors listed the time they spent

on a variety of tasks during the two working days prior to our survey.15

3.2 Experiment

Our experimental sample comprises 35 of the 36 districts in Punjab. We remove Khanewal

from the experimental sample as that district served as the location for our pilot. We

randomly implemented the smartphone program in 18 of the 35 remaining districts. We

randomized at the district level for two reasons. First, the intervention channels information

about health inspections to district-level senior health officials. Second, all inspectors in

a district are required to attend monthly meetings and so interact frequently, while these

relations are much weaker across districts. District-level randomization therefore makes sense

in terms of the design of program and also reduces concerns about contamination.

13Doctors are officially required to be present and see patients at the health clinic. An unannounced visittherefore captures the official work assigned to doctors. We did not capture data on computer techniciansas they are rarely assigned.

14In some cases enumerators were unable to confidently verify whether or not an inspection had occurredin the prior month. We treat such cases as missing data for analysis and verify in Appendix Table A1 thatsuch cases do not correlate with treatment assignment.

15Inspectors picked up to three out of 10 possible categories of work to account for each hour between8am and 6pm. In addition, they were asked to identify when they arrived for, and left from work. Wealso collected additional information from health inspectors during these interviews, and we also intervieweddoctors and senior health officials across our sample through similar face-to-face interviews. We will brieflydiscuss some of the other data collected through these interviews in Section 4.6 below.

9

We stratified treatment on baseline health clinic staff attendance, the number of clinics in

a district (to ensure a roughly even number of clinics in treatment and control), and whether

or not the district was being run by the World Bank-led Public Sector Reform Program

(PSRP). Figure 3 depicts control and treatment districts. We then re-randomized to achieve

balance on a set of variables related to health clinic staff attendance, the frequency of health

inspections, and the quality of service provision.16

Appendix Table A1 reports balance between clinics in treatment and control districts at

the baseline. The only measure showing imbalance is an indicator for doctor presence during

our unannounced visit. While stratifying on the share of staff present at baseline achieved

balance for five of the six categories of staff that are supposed to be present at health clinics,

it did not do so for doctors. We therefore use a difference-in-differences specification to

estimate impacts of the program on doctors. Appendix Figure A1 reports a long time series

of administrative data on doctor attendance from paper records, showing that these trends

were parallel in the pre-treatment period.

4 Results

We now present results from our experimental evaluation of the ‘Monitoring the Monitors’

program. We estimate treatment effects using the following specification:

Yitds = α + βTreatmentd + εitds (1)

16Specifically, we randomized using the ‘big stick’ approach, whereby we redrew our treatment assignmentuntil the minimum p-value from difference in means tests between health clinics in treatment and controldistricts for a pre-specified set of balance variables was greater than some threshold. In our case we selecteda threshold of 0.21. We balanced on the share of assigned health clinic staff who were present during thebaseline unannounced visit, whether the health clinic had been inspected in the previous month by its healthinspector, whether the health clinic had been inspected in the previous month by its senior health official,the number of antenatal visits recorded on the health clinic register in the previous month, the log of thenumber of polio vaccines administered at the health clinic in the previous month, whether the health clinic’sdoctor claimed a connection to their local parliamentarian, the tenure of the health clinic’s doctor, the logof the population in the health clinic’s catchment area, and the log of the distance of the health clinic to thedistrict’s headquarters.

10

Yitds either a dummy equal to one for health clinics inspected in the previous month, a

measure of health inspector time use, or a measure of health clinic staff attendance, where

i refers to the clinic, t to the survey wave, d refers to the district, and s refers to the

randomization strata that the district was in. Treatmentd is a dummy variable equal to 1

for treated districts. These regressions use only post-treatment data (survey waves 2 and

3). We cluster all standard errors at the district level. We also estimate impacts using the

difference-in-differences specification:

Yitds = α + β1Treatmentd + β2Postt + β3Treatd ∗ Postdt + δt + γi + εitds, (2)

where δt are survey wave fixed effects and γi are clinic fixed effects.

4.1 Approach to Inference

With only 35 districts in our sample, the asymptotic reference distributions for our test

statistics may be invalid. We therefore report Fisher exact p-values (Fisher, 1935) which do

not require a limiting distribution (Gerber and Green, 2012). This test assumes a null of no

treatment effect for any unit. We perform this test by creating a set of artificial treatment

assignments that satisfy the balancing requirements of the assignment protocol for actual

treatment. For each treatment assignment, a corresponding artificial treatment effect is

generated. The effect estimated using the actual treatment assignment is then compared

against the 1,000 artificial treatment effect estimates. The p-value is the share of artificial

treatment effects that have a larger magnitude than the actual treatment effect.

4.2 Impact on Health Inspectors

Table 1 presents estimates of the program’s impact on the rate of inspections. We find that

health clinics in treatment districts were 18.1 percentage points more likely to be inspected

in the previous month during the treatment period. This represents a 74 percent increase

11

Table 1: The Effect of Smartphone Monitoring on the Rate of Inspections

P-value P-valueTreatment Control Difference Mean Diff. Exact Test

(1) (2) (3) (4) (5)

Facility Inspected in the Previous Month (=1) 0.426 0.245 0.181 0.010 0.001(0.048) (0.046) (0.066)

# of Observations 759 764

Facility Inspected (=1), Wave 2 Only (June 2012) 0.519 0.255 0.264 0.002 0.003(0.063) (0.048) (0.079)

# of Observations 366 373

Facility Inspected (=1), Wave 3 Only (October 2012) 0.338 0.235 0.103 0.200 0.065(0.053) (0.059) (0.079)

# of Observations 393 391

Notes: This table reports unconditional average treatment effects of the ‘Monitoring the Monitors’ program on the rate of health clinic

inspections. The unit of observation in is the health clinic. The data come from primary unannounced surveys after the treatment was

launched (wave 2 and 3). The dependent variable is an indicator variable that equals 1 if an inspector visited a clinic within a month prior

to the survey, and 0 otherwise. The regression reports differences between treatment and control clinics. P-values reported in column (4)

are for the difference in mean between columns (1) and (2) (i.e. that the treatment had no impact). Column (5) reports the Fisher Exact

p-values. Standard errors clustered at the district level are reported in parentheses. Results conditional on randomization strata fixed

effects and using a difference-in-differences specification be found in Appendix Table A3.

in inspection rates in treatment districts relative to control districts. Breaking this up into

the two waves of post-treatment data collection, we find comparable effects, though there

is evidence that the effect of treatment had attenuated by October 2012, a year after the

introduction of the program.17

We examine whether the additional time for inspections come at the costs of other tasks

(although inspectors are almost exclusively tasked with performing inspections). Appendix

Table A2 reports the monitoring program’s impact on the time use of health inspectors. We

do not find any evidence that health inspectors in treatment districts are spending less time

on other tasks after treatment, while they do seem to be spending more time inspecting

health clinics. However, we treat these results speculatively, as we only have a sample of 117

inspectors and a noisy measure of time use.

17The p-value corresponding to the test of equal treatment effects in wave 2 and in wave 3 is 0.08.

12

4.3 The Impact of ‘Monitoring the Monitors’ on Health Clinic

Staff

Our estimates indicate that the program increased health inspections; this could increase

health clinic staff attendance (an explicit goal of the program). We report estimates of

program impact on staff attendance in Appendix Table A3, using a difference-in-differences

specification to account for the baseline imbalance in doctor attendance. We find that the

program had no effect on attendance for any category of staff.18

In section 4.6 below, we provide speculative evidence that the program had positive ef-

fects on doctor attendance in politically competitive districts, a dimension of heterogeneity

forecasted by the policymakers while implementing the program. In addition, there is specu-

lative evidence that the program had positive effects on the attendance of doctors that have

more ‘positive’ personality characteristics, consistent with a small but growing literature on

the role of individual characteristics in public service delivery.

4.4 Do Policy Makers Use Data on Staff Attendance?

The system aggregates and presents data to senior health officials through an online dash-

board. In addition to these officials, this dashboard is visible to the Health Secretary and

the Director General of Health for Punjab.

To test whether senior health officials would act on these data, we introduced a manipula-

tion to the dashboard that made certain health clinic inspection reports salient. Specifically,

we highlighted in red inspection reports that reported three or more staff (of 7 generally) as

absent during a health inspector’s visit to the clinic. Figure 2 Panel B provides an exam-

ple of a dashboard view with some facilities highlighted in red. The exact formula for this

arbitrary threshold was not known to anyone but the research team.

18Staff assignment to health clinics may itself be affected by treatment. However, assignment changeswere purposefully suppressed by the government during the experiment. We check this by testing if ourtreatment had an impact on the probability of staff assignment. We do not find any impacts. See AppendixTable A5.

13

4.5 Dashboard Manipulation Impacts

We examine whether this manipulation affected subsequent doctor absence in our primary

data with the following specification:

Present Surveyjt = α + β1Flaggedjt−1 + δt + ηjt (3)

Present Surveyjt is a dummy variable equal to 1 if the doctor j was absent during an

unannounced visit by our enumerator in wave t, Flaggedit−1 is a dummy variable that

equals 1 if the facility was flagged in red on the dashboard in a window of time prior to the

primary survey wave t. For our primary analysis, we define this window as 11 to 25 days

before an unannounced visit by our field enumerators. We discuss the robustness of our

result to the selection of time windows below. To minimize concerns related to fundamental

differences between facilities that were flagged and not flagged, and thus to isolate the effect

of the flagging itself, we restrict our sample to only facility reports at the flagging threshold

(facilities with two or three staff absent).

Table 2 reports results from this test for the various staff posted to a clinic. We observe

a positive effect of making staff absence salient on subsequent doctor attendance, which

increases by 17.7 percentage points or about 75 percent.19 In contrast to subsequent doctor

attendance however, attendance for other staff does not respond to getting flagged on the

dashboard. One reason for this could be, speculatively, that attendance for other staff is

much higher at baseline compared to doctors so a null effect of the treatment on them could

indicate that there are ceiling effects of the flagging treatment. Doctors are also in-charge

of the facilities so it makes sense that they are the ones to respond to potential censure

from higher level bureaucrats. Finally, some staff, like the School Health and Nutrition

19Note that this positive result is not at odds with the lack of an average treatment effect of ‘Monitoringthe Monitors’ on doctor attendance. To detect the effect of flagging we are limiting our sample to treatmentfacilities only (those that could be flagged on the dashboard) that were right below or above the staff absencecutoff for flagging. If we average our estimated flagging effect of 0.177 over all post-treatment observationswithin treatment districts (850), we obtain 0.027 which is not statistically different from our estimatedaverage treatment effects of ‘Monitoring the Monitors’.

14

Table 2: Effect of Flagging Underperformance on the Dashboard

Doctor Dispenser LHW Health Tech SHNS Midwife(1) (2) (3) (4) (5) (6)

Flagged 0.177 0.076 -0.115 -0.066 -0.037 -0.125(0.082) (0.083) (0.090) (0.092) (0.093) (0.091)

Unflagged Mean 0.236 0.676 0.703 0.405 0.324 0.568# Clinics 112 116 116 116 116 116# Reports 130 136 136 136 136 136R-Squared 0.298 0.258 0.190 0.264 0.176 0.242District Fixed Effects Yes Yes Yes Yes Yes Yes

Notes: This table reports on the effect on subsequent doctor attendance of flagging on an online dash-

board the fact that a clinic had three or more staff absent to a senior policymaker. Clinics were flagged in

red on an online dashboard if three or more of the seven staff were absent in one or more health inspections

of the clinic 11 to 25 days prior to an unannounced visit by our survey enumerators. The Discontinuity

sample limits to facility reports in which either two or three staff were absent (the threshold to trigger the

underreporting red flag). In addition, the sample in all columns is limited to ’Monitoring the Monitors’

treatment districts due to the necessity of the web dashboard for flagging clinics. All regressions include

survey wave fixed effects. LHWs are Lady Health Workers and SHNS are School Health and Nutrition

Supervisors. Standard errors clustered at the clinic level reported in parentheses.

Supervisor and Lady Health Workers, have their primary job outside of the specific facility

where attendance was audited. Their attendance is therefore unlikely to be affected by these

reports.

Selecting the window for which Flaggedit−1 is equal to 1 involves trade-offs and provides

us substantial discretion in the analysis. We expect it would take at least a few days for

senior officials to acknowledge and act on data from the dashboard. At the same time, if

the window for which Flaggedit−1 is set to 1 is too long, virtually every facility will become

flagged and we will lose variation. We find that the estimated effect on doctor attendance is

robust to a broad number of potential time windows.

We test the robustness of the results of Table 2 in Appendix Figure A2. We do this

by running the same regression 750 times varying the window for which we define a clinic

as flagged prior to a primary unannounced visit to a clinic along two dimensions—we vary

the length of the window being used along the x-axis and the delay from when a clinic is

highlighted in red to when the window begins along the y-axis (so for example, a length of

25 and delay of 15 corresponds to considering a clinic as flagged if it was highlighted in red

15

anytime 15 to 40 days prior to an unannounced visit). For each window, we report using a

colored pixel the t-statistics of the hypothesis test of a null effect of flagging on subsequent

doctor attendance. We observe a positive, robust, and significant treatment effect of flagging

on doctor attendance across a wide range of windows.

Placebo Tests: As an additional robustness check, in Appendix Figure A3, we run the

same hypothesis tests as are conducted in Table 2 using placebo thresholds between one and

two absences and between three and four absences on the dashboard. We are not able to

reject the null hypothesis of no effect on this placebo flagging in nearly all cases.

4.6 Impact Heterogeneity

Whether systems like the one introduced in Punjab affect service delivery likely depends on

the abilities of government personnel and also the broader political environment. The scale

of our experiment provides scope to examine two features of the environment.20

First, we seek to understand whether the individual characteristics of the health work-

ers in our study matter for their performance on the job under status quo incentives (i.e.

at baseline) and for their responses to monitoring or to changes in information salience.21

To do this, we measured personality characteristics—the Big 5 Personality Index and the

Perry Public Service Motivation Index—of a large, representative sample of doctors from

our 850 sample clinics as well as the universe of health inspectors and senior health officials

in Punjab. We find that personality characteristics systematically predict doctor attendance

at baseline, responses by health inspectors to ‘Monitoring the Monitors’, and responses by

senior health officials to our dashboard experiment, as measured by future doctor attendance

in flagged facilities. Doctors who score one standard deviation higher on the measured Big

5 trait of conscientiousness, for example, are 5.8 percentage points more likely to be present

at work during an unannounced visit. In all three cases, personality characteristics predict

20The results in this section are explored in much greater detail in two reports: Callen et al. (2017a) andCallen et al. (2017b).

21A growing literature documents the role of individuals in public service delivery across the developingworld, summarized in Finan et al. (2017).

16

performance and response to treatment at least as well as other demographic variables avail-

able. Of course, these results are merely predictive as we do not have identifying variation

in personality traits. See Callen et al. (2017a) for details.

Second, we explore if political factors predict the efficacy of the program. The bureaucrats

we worked with to create the program felt strongly that the program would break down if

politicians interfered when senior officials attempted to sanction their subordinates. Indeed,

in our surveys, senior health officials report that politicians routinely interfere in this way.22

We match each clinic in our sample to a provincial assembly constituency and study if the

treatment effect on doctor attendance varies by the degree of competitiveness in the previous

election. We find that while the program increases health inspections uniformly across

constituencies, doctors only respond to the increase in inspections in the most politically

competitive tercile of constituencies, where their attendance probability increases by 10.2

percentage points (standard error = 6.3pp). We also check if the effect of flagging a facility

on the online dashboard changes by the moderators mentioned above. We find that while

increases subsequent attendance increases by 32 percentage points in the most politically

competitive third of constituencies, flagging has no apparent effect in the least competitive

tercile. In addition, flagging seems to work better on doctors who do not report a direct

connection with a local politician.23

5 Discussion and Conclusion

A fundamental objective of policy research is to convey facts and data to policy makers. We

find that providing senior health officials with information on low staff attendance causes

them to take corrective action, indicating that data can change how policy makers behave,

even in settings with weak institutions. This suggests two additional general lines of inquiry.

First, our test of whether information affects policy decisions is direct. For our senior

22Based on interviews with all senior health officials in Punjab, we find that forty-four percent report apolitician interfering in their decision to sanction an underperforming employee during the previous year.

23See Callen et al. (2017b) for further details

17

officials, who are tasked with responsibilities that affect millions, ensuring the functioning

of frontline service facilities is a priority. A natural question is whether we would see the

same response to a more complicated object, like causal estimates of program effect, cost-

benefit analyses, or more general forms of research evidence. The ‘Monitoring the Monitors’

program provides proof-of-concept that technology can mobilize data to real effect. This also

suggests, but by no means proves conclusively, that the massive investments being made by

governments, technologists, researchers, philanthropists, and aid organizations to promote

evidence-based policy can make a difference in developing countries.

Second, whether data and evidence impact policy likely depends on the characteristics of

policy makers, and the political and institutional environment in which they operate. The

scale of our experiment provides enough variation to examine preliminarily whether these

factors matter. We find some evidence, albeit highly speculative, that these considerations

are relevant. The personalities of both inspectors and senior health officials predict how the

‘Monitoring the Monitors’ program impacted them. Similarly, the degree of local political

competition predicts where the program will work best. These results are correlational, and

should be treated with appropriate skepticism, but do suggest that the effect of data on

policy decision might depend critically on context.

Developing countries are witnessing a revolution in the amount of data that can quickly

and cheaply be accessed for policy decisions. In our case, the ‘Monitoring the Monitors’

system involved a fixed cost of about 17,800 USD to set up, and a monthly cost of 510 USD

to operate.24 This trend is only likely to accelerate with the rise of remote sensing, digital

trace (e.g., cell phone call and mobile money transaction records), smartphones, and other

research innovations. A key lesson of this experiment is that, appropriately channeled, these

data streams can improve policy outcomes.

24The set up costs included about 4,470 USD to develop the app and about 13,330 USD for smartphones.

18

References

Ashraf, Nava, Oriana Bandiera, and Kelsey Jack, “No Margin, No Mission? A Field

Experiment on Incentives for Public Services Delivery,” Journal of Public Economics,

2014, 120, 1–17.

, , and Scott Lee, “Do-gooders and Go-getters: Career Incentives, Selection, and

Performance in Public Service Delivery.,” 2015.

Banerjee, Abhijit and Esther Duflo, “Addressing Absence,” The Journal of Economic

Perspectives, 2006, 20 (1), 117–132.

Banerjee, Abhijit V., Esther Duflo, and Rachel Glennerster, “Putting a Band-Aid

on a Corpse: Incentives for Nurses in the Indian Public Health Care System,” Journal of

the European Economic Association, 04-05 2008, 6 (2-3), 487–500.

Bertrand, Marianne, Robin Burgess, Arunish Chawla, and Guo Xu, “The Glitter-

ing Prizes: Career Incentives and Bureaucratic Performance,” Working Paper, 2017.

Blum, Florian and Rohini Pande, “Data poverty makes it harder to fix real poverty.

That’s why the UN should push countries to gather and share data.,” July 2015.

Callen, Michael and James D Long, “Institutional Corruption and Election Fraud:

Evidence From a Field Experiment in Afghanistan,” The American Economic Review,

2015, 105 (1), 354–381.

, Clark Gibson, Danielle Jung, and James Long, “Improving Electoral Integrity with

Information and Communications Technology,” Journal of Experimental Political Science,

2016, 3 (1), 4–17.

, Saad Gulzar, Ali Hasanain, Yasir Khan, and Arman Rezaee, “Personalities

and Public Sector Performance: Evidence from a Health Experiment in Pakistan,” 2017.

NBER Working Paper #21180.

19

, , , , and , “The Political Economy of Public Sector Absence,” 2017. NBER

Working Paper #22340.

Chaudhury, Nazmul, Jeffrey Hammer, Michael Kremer, Karthik Muralidharan,

and F. Halsey Rogers, “Missing in Action: Teacher and Health Worker Absence in

Developing Countries,” Journal of Economic Perspectives, Winter 2006, 20 (1).

Dal Bo, Ernesto, Frederico Finan, and Martın A. Rossi, “Strengthening State Ca-

pabilities: The Role of Financial Strengthening State Capabilities: The Role of Financial

Incentives in the Call to Public Service,” Quarterly Journal of Economics, 2013, 128 (3),

1169 – 1218.

Deserranno, Erika, “Financial Incentives as Signals: Experimental Evidence from the

Recruitment of Village Promoters in Uganda,” 2017.

Dhaliwal, Iqbal and Rema Hanna, “Deal with the Devil: The Successes and Limitations

of Bureaucratic Reform in India,” Journal of Development Economics, 2017.

Finan, Frederico, Benjamin A Olken, and Rohini Pande, The Personnel Economics

of the State, North Holland, 2017.

Fisher, Ronald, The Design of Experiments, Edinburgh: Oliver and Boyd, 1935.

Gerber, Alan S. and Donald P. Green, Field Experiments: Design, Analysis, and

Interpretation, New York, NY: Norton New York, 2012.

Gulzar, Saad and Benjamin J. Pasquale, “Politicians, Bureaucrats, and Development:

Evidence from India,” American Political Science Review, 2017, 111 (1), 162–183.

Khan, Adnan, Asim Ijaz Khwaja, and Benjamin A. Olken, “Tax Farming Redux:

Experimental Evidence on Performance Pay for Tax Collectors,” Quarterly Journal of

Economics, 2016, 131 (1), 219–271.

20

Muralidharan, Karthik and Paul Niehaus, “Experimentation at Scale,” 11 2017, 31,

103–124.

, , and Sandip Sukhtankar, “Building state capacity: Evidence from biometric smart-

cards in India,” The American Economic Review, 2016, 106 (10), 2895–2929.

Nealer, Erin, Charles Rice, and Erol Yayboke, “Harnessing the Data Revolution to

Achieve the Sustainable Development Goals: Enabling Frogs to Leap,” Technical Report,

Center for Stategic and International Studies 2017.

Rogger, Daniel and Imran Rasul, “Management of Bureaucrats and Public Service

Delivery: Evidence from the Nigerian Civil Service,” Economic Journal, forthcoming.

Xu, Guo, “The Costs of Patronage: Evidence from the British Empire,” 2017.

21

APPENDIX: FOR ONLINE PUBLICATION ONLY

A Additional Tables and Figures

A.1 Tables

Table A1: Randomization Verification

Conventional Smartphone Difference P-value Control TreatmentMonitoring (=1) Monitoring (=1) Observations Observations

Health clinic open during visit (=1) 0.926 0.930 -0.004 0.911 420 428[0.262] [0.256] (0.033)

Number of Staff Assigned to the Health Clinic 5.121 5.285 -0.164 0.185 420 428[0.925] [0.940] (0.121)

Number of Staff Present 2.719 2.883 -0.164 0.370 420 428[1.514] [1.637] (0.181)

Doctor Present (=1) 0.237 0.408 -0.171 0.003 409 414[0.426] [0.492] (0.054)

Health/Medical Technician Present (=1) 0.396 0.349 0.047 0.392 409 413[0.490] [0.477] (0.055)

Dispenser Present (=1) 0.706 0.777 -0.071 0.259 408 413[0.456] [0.417] (0.062)

School Health and Nutrition Supervisor Present (=1) 0.347 0.341 0.006 0.922 406 413[0.477] [0.475] (0.060)

Lady Health Worker Present (=1) 0.581 0.634 -0.054 0.298 408 413[0.494] [0.482] (0.051)

Midwife Present (=1) 0.536 0.478 0.058 0.206 405 412[0.499] [0.500] (0.045)

Health Inspector Has Visited in the Last Month (=1) 0.229 0.219 0.010 0.855 332 320[0.421] [0.414] (0.055)

Enumerator Could Not Verify Last Health Inspection (=1) 0.210 0.252 -0.043 0.467 420 428[0.407] [0.435] (0.058)

Senior Health Official Has Visited in the Last Month (=1) 0.041 0.038 0.003 0.883 246 290[0.198] [0.191] (0.018)

Number of Antenatal Visits in the Last Month 54.255 59.115 -4.859 0.601 274 288[54.344] [49.144] (9.217)

Number of Polio Vaccinations in the Last Month 480.226 649.748 -169.522 0.542 305 326[1130.822] [1665.478] (274.994)

Doctor Connected to Local Parliamentarian (=1) 0.330 0.273 0.057 0.542 91 176[0.473] [0.447] (0.092)

Doctor’s Tenure (in months) 111.732 106.025 5.707 0.779 82 161[98.782] [90.214] (20.156)

Population of Health Clinic’s Catchment Area 22.159 24.243 -2.084 0.187 416 426[6.934] [8.161] (1.546)

Distance of the Health Clinic to the District’s HQ (kms) 49.521 49.464 0.058 0.992 420 401[29.334] [30.817] (5.563)

Notes: This table checks balance between treatment and control clinics. The unit of observation is the clinic (basic health unit). The first ten rows report data from the baseline

survey of health facilities which involved making unannounced visits to facilities in November, 2011. The last four rows report data based on the February 2008 parliamentary

election. The political competition index is a Herfindahl index computed as the sum of squared candidate vote shares in each provincial assembly constituency. Variable standard

deviations are reported in brackets. Standard errors are reported in parentheses.

22

Tab

leA

2:T

he

Eff

ect

ofSm

artp

hon

eM

onit

orin

gon

Insp

ecto

rT

ime-

Use

Insp

ecti

ng

Man

agem

ent

inH

ead

Offi

ceO

ther

Wor

kT

otal

Tot

alT

otal

Imm

uniz

atio

nO

ffici

alO

ther

Tot

alW

ork

Clinic

sH

ospit

als

(1)+

(2)

Clinic

sH

ospit

als

(4)+

(5)

Cam

pai

gns

Mee

tings

Wor

k(8

)+(9

)(3

)+(6

)+(1

0)B

reak

s(1

)(2

)(3

)(4

)(5

)(6

)(7

)(8

)(9

)(1

0)(1

1)(1

2)

Panel

A:

Wit

hout

Blo

ckF

ESm

artp

hon

eM

onit

orin

g15

.952

-13.

042

2.91

0-5

.833

-9.0

67-1

4.90

111

.620

35.5

95-1

.389

45.8

2733

.836

-2.8

57(1

3.09

1)(1

3.53

9)(1

6.52

9)(9

.786

)(1

6.36

5)(1

6.72

1)(2

5.28

0)(2

4.78

9)(2

9.77

4)(3

6.47

5)(3

0.75

4)(6

.136

)E

xac

tp-v

alue

[0.1

01]

[0.8

27]

[0.5

20]

[0.5

44]

[0.7

40]

[0.8

31]

[0.3

64]

[0.1

25]

[0.5

36]

[0.1

46]

[0.1

49]

[0.4

18]

Con

trol

Mea

n49

.444

47.9

6397

.407

37.5

0040

.417

77.9

1796

.713

66.9

4462

.500

226.

157

401.

481

24.4

44#

Dis

tric

ts35

3535

3535

3535

3535

3535

35#

Obse

rvat

ions

117

117

117

117

117

117

117

117

117

117

117

117

Panel

B:

Wit

hB

lock

FE

Sm

artp

hon

eM

onit

orin

g23

.668

-13.

616

10.0

52-8

.025

6.49

5-1

.530

3.36

736

.679

-4.2

9735

.749

44.2

71-2

.821

(9.5

62)

(11.

618)

(11.

419)

(8.8

10)

(9.8

23)

(12.

257)

(24.

837)

(20.

898)

(33.

460)

(33.

915)

(30.

900)

(5.5

54)

Exac

tp-v

alue

[0.0

34]

[0.8

91]

[0.2

51]

[0.4

82]

[0.3

24]

[0.2

61]

[0.4

89]

[0.1

94]

[0.6

85]

[0.3

02]

[0.1

97]

[0.3

79]

Con

trol

Mea

n49

.444

47.9

6397

.407

37.5

0040

.417

77.9

1796

.713

66.9

4462

.500

226.

157

401.

481

24.4

44#

Dis

tric

ts35

3535

3535

3535

3535

3535

35#

Obse

rvat

ions

117

117

117

117

117

117

117

117

117

117

117

117

Notes:

Th

ista

ble

rep

ort

sav

erag

etr

eatm

ent

effec

tsof

the

‘Mon

itor

ing

the

Mon

itors

’p

rogr

amon

the

staff

atte

nd

ance

usi

ng

ad

iffer

ence

-in

-diff

eren

ces.

Th

eu

nit

of

ob

serv

ati

on

isth

ecl

inic

,an

dd

ata

com

e

from

pri

mar

yu

nan

nou

nce

dsu

rvey

saf

ter

the

trea

tmen

tw

asla

un

ched

.T

he

dep

end

ent

vari

ab

leis

an

ind

icat

orfo

rw

het

her

anass

ign

edd

oct

orw

asp

rese

nt

at

the

clin

icd

uri

ng

an

an

nou

nce

dvis

it.

Sta

nd

ard

erro

rscl

ust

ered

atth

ed

istr

ict

leve

lar

ein

par

enth

eses

.S

quar

eb

rack

ets

rep

ort

the

Fis

her

Exac

tp

-val

ues

.

23

Table A3: The Effect of Smartphone Monitoring on Inspections

Wave2+3 Wave 2 Wave 3 Wave2+3 Wave 2 Wave 3 Diff-in-Diff Diff-in-Diff(1) (2) (3) (4) (5) (6) (7) (8)

Smartphone Monitoring 0.181 0.264 0.103 0.161 0.261 0.058 0.181 0.210(0.066) (0.079) (0.079) (0.056) (0.063) (0.073) (0.066) (0.065)

Exact p-value [0.002] [0.004] [0.066] [0.009] [0.014] [0.191] [0.002] [0.000]

Control Mean 0.245 0.255 0.235 0.245 0.255 0.235 0.240 0.239# Districts 35 35 35 35 35 35 35 35# Observations 1523 739 784 1523 739 784 2175 2117Fixed Effects No No No Block Block Block Wave Wave+Facility

Notes: This table reports average treatment effects of the ‘Monitoring the Monitors’ program on the staff attendance using a difference-

in-differences. The unit of observation is the clinic, and data come from primary unannounced surveys after the treatment was launched.

The dependent variable is an indicator for whether an assigned doctor was present at the clinic during an announced visit. Standard errors

clustered at the district level are in parentheses. Square brackets report the Fisher Exact p-values.

24

Table A4: The Effect of Smartphone Monitoring on Clinic Staff Attendance

Lady Health Health School HealthDoctor Dispenser Visitor Tech Nutri. Supervisor Midwife

(1) (2) (3) (4) (5) (6)

Panel A: Survey Wave FEMonitoring -0.018 -0.064 -0.080 0.030 0.012 0.055

(0.043) (0.061) (0.053) (0.058) (0.072) (0.065)[0.692] [0.945] [0.990] [0.511] [0.537] [0.435]

Mean in Controls 0.227 0.736 0.594 0.413 0.318 0.535# Districts 35 35 35 35 35 35# Observations 2422 2457 2457 2458 2448 2448R-Squared 0.032 0.003 0.005 0.004 0.004 0.002

Panel B: Survey Wave + Facility FEsMonitoring -0.010 -0.067 -0.075 0.028 0.013 0.055

(0.043) (0.059) (0.054) (0.059) (0.072) (0.064)[0.645] [0.968] [0.942] [0.465] [0.701] [0.386]

Mean in Controls 0.228 0.736 0.596 0.413 0.318 0.537# Districts 35 35 35 35 35 35# Observations 2408 2446 2446 2447 2437 2437R-Squared 0.533 0.464 0.447 0.552 0.398 0.512

Notes: This table reports average treatment effects of the ‘Monitoring the Monitors’ program on the staff

attendance using a difference-in-differences specification. The unit of observation is the clinic, and data come

from primary unannounced surveys after the treatment was launched. The dependent variable is an indicator for

whether a doctor was present at the clinic during an announced visit. Standard errors clustered at the district

level are in parentheses. Square brackets report the Fisher Exact p-values.

25

Table A5: The Effect of Smartphone Monitoring on Clinic Staff Assignment

Lady Health Health School HealthDoctor Dispenser Visitor Tech Nutri. Supervisor Midwife

(1) (2) (3) (4) (5) (6)

Panel A: Survey Wave FEMonitoring 0.007 -0.021 -0.038 -0.061 -0.014 -0.061

(0.049) (0.016) (0.023) (0.053) (0.118) (0.068)Exact p-value [0.457] [0.889] [0.942] [0.881] [0.395] [0.833]Mean in Controls 0.532 0.979 0.938 0.749 0.876 0.855# Districts 35 35 35 35 35 35# Observations 2422 2457 2457 2458 2479 2483R-Squared 0.046 0.004 0.002 0.013 0.077 0.107

Panel B: Survey Wave + Facility FEsMonitoring 0.023 -0.024 -0.033 -0.051 -0.010 -0.061

(0.048) (0.016) (0.024) (0.052) (0.117) (0.068)Exact p-value [0.441] [0.878] [0.952] [0.965] [0.380] [0.840]Mean in Controls 0.533 0.979 0.937 0.748 0.876 0.854# Districts 35 35 35 35 35 35# Observations 2408 2446 2446 2447 2473 2477R-Squared 0.677 0.521 0.551 0.648 0.558 0.504

Notes: This table reports average treatment effects of the ‘Monitoring the Monitors’ program on the staff

assignment using difference-in-differences. The unit of observation is the clinic, and data come from primary

unannounced surveys after the treatment was launched. The dependent variable is an indicator for whether a

staff member was assigned at the clinic during an unannounced visit. Standard errors clustered at the district

level are in parentheses. Square brackets report the Fisher Exact p-values.

26

A.2 Figures

0.2

.4.6

.81

Doc

tor P

rese

nce

in A

dmin

istra

tive

Dat

a

July

201

1

Nov

embe

r 201

1

Mar

ch 2

012

July

201

2

Control Treatment

Figure A1: Time-trend on Doctor Attendance Before and After Treatment

27

+0

+5

+10

+15

+20

+25

0 10 20 30

-2.58-1.96-1.65

0

1.651.962.58

t-stats

Doctor

+0

+5

+10

+15

+20

+25

0 10 20 30

-2.58-1.96-1.65

0

1.651.962.58

t-stats

Dispenser

+0

+5

+10

+15

+20

+25

0 10 20 30

-2.58-1.96-1.65

0

1.651.962.58

t-stats

Lady Health Visitor

+0

+5

+10

+15

+20

+25

0 10 20 30

-2.58-1.96-1.65

0

1.651.962.58

t-stats

Health Tech.

+0

+5

+10

+15

+20

+25

0 10 20 30

-2.58-1.96-1.65

0

1.651.962.58

t-stats

School Health Nutri. Sup.

+0

+5

+10

+15

+20

+25

0 10 20 30

-2.58-1.96-1.65

0

1.651.962.58

t-stats

Midwife

Leng

th o

f ana

lysi

s w

indo

w (

days

)

Days since dashboard report

Figure A2: Absence after Flagging

Notes: This figure presents robustness of flagging results in Table 2 to the window of time prior to an

unannounced visit that a clinic being highlighted in red on the dashboard is considered flagged. Each figure

reports t-stats from 1300 hypothesis tests analogous to that conducted in Table 2 column (3) that Flagged

= 0, varying the window for which we define a clinic as flagged prior to a primary unannounced visit to a

clinic along two dimensions—we vary the length of the window being used along the x-axis and the delay

from when a clinic is highlighted in red to when the window begins along the y-axis (so for example, a length

of 30 and delay of 15 corresponds to considering a clinic as flagged if it was highlighted in red anytime 15 to

45 days prior to an unannounced visit).

28

+0

+5

+10

+15

+20

+25

0 10 20 30 40 50

-2.58

-1.96-1.65

0

1.651.96

2.58

t-stats

1/2 Cutoff Placebo

+0

+5

+10

+15

+20

+25

0 10 20 30 40 50

-2.58

-1.96-1.65

0

1.651.96

2.58

t-stats

3/4 Cutoff Placebo

Leng

th o

f ana

lysi

s w

indo

w (

days

)

Days since dashboard report

Figure A3: Absence after Flagging Placebos

Notes: This figure presents placebo tests for the flagging results in Table 2, coding the data as if the cutoff

for a health clinic being highlighted in red was for more than 1 staff absent or more than 3 staff absent,

rather than the true cutoff that was used (more than 2 staff absent). Each figure reports t-stats from

1300 hypothesis tests analogous to that conducted in Table 2 column (3) that Flagged = 0, with these

placebo definitions of flagging, varying the window for which we define a clinic as flagged prior to a primary

unannounced visit to a clinic along two dimensions—we vary the length of the window being used along the

x-axis and the delay from when a clinic is highlighted in red to when the window begins along the y-axis (so

for example, a length of 30 and delay of 15 corresponds to considering a clinic as flagged if it was highlighted

in red anytime 15 to 45 days prior to an unannounced visit).

29

30

BP

ap

er

Insp

ect

ion

Form

INSPEC

TION FORM

 OF BA

SIC HEA

LTH UNIT 

HEA

LTH DEP

ART

MEN

T (GOVER

NMEN

T OF TH

E PU

NJAB) 

A ‐ BA

SIC HEA

LTH UNIT IN

FORM

ATION  

HMIS Cod

e:  

  

  

Nam

e of BHU: ______________________________________________ 

Managed

 by: i) PRSP 

 ii) Dist. Govt.   

 (please tick on

ly one

 option) 

Mauza: _

_________________________ UC Nam

e: _______________________________

 UC No. ___________________

 

NA No. _________

 PP. No. _________

 District: _________________________

 Teh

sil: _____________________________ 

Nam

e of incharge 

of the

 facility: __________________________________

 Designatio

n: _

_____________________ 

Mob

ile N

o.: _______________________ 

BHU’s P

hone

 (with

 cod

e): _______________________________________ 

Nam

e of DDOH: _

____________________________________________

 Referen

ce No. ___________________________

 

Date & Tim

e of arrival fo

r inspectio

n: _____/_______/______  Tim

e: Hou

rs __________ Minutes _______

       am    / pm

  

B ‐ C

LEANLINESS AND GEN

ERAL  

OUTLOOK OF TH

E FA

CILITY

 (tick relevant colum

n) 

C ‐ D

ISPLAYS

 (tick relevant colum

n) 

Sr. N

o Location

 Goo

d Average 

Poor 

ITEM

S Ye

s No 

1 Bo

undary W

all 

  

 Signbo

ards/Directio

n Bo

ard displayed 

  

2 Lawns 

  

   Display in

 the MO/ 

Incharge 

office: 

1) Organogram 

  

3 Waitin

g Area 

  

 2) M

ap o

f Union

 cou

ncil show

ing all 

localities 

  

4 Bu

ilding 

  

  

 5 

Labo

ur Roo

m 

  

 3) Statistics of the

 Union

 Cou

ncil and 

the BH

U 

  

6 Wards 

  

  

 7 

Toilets 

  

 4) Tou

r Programme of ‘outreach team

’  

  

D ‐ AVAILBILITY

 OF UTILITIES 

(tick relevant colum

n) 

E ‐D

ISPO

SAL OF HOSPITAL WASTE

(tick relevant colum

n) 

Sr. No 

   

Nam

e of Utility 

   

Not 

Available 

Available 

Sr. N

o Mod

e Ye

s No 

Function

al 

Non

 Function

al 

1 Hospital Waste Segregated

 as pe

r guidelines 

  

1 Electricity

  

  

2 Hospital W

aste Lying

 Ope

n  

 2 

Teleph

one 

  

 3 

  Burnt b

y: 

(a) Incinerator 

  

3 Water sup

ply System

  

  

(b) O

ther m

eans 

  

4 Sui G

as 

  

 4 

Buried

  

 5 

Sewerage System

  

  

5 Carried aw

ay by mun

icipality

  

 6 

Other 

  

  

6 Any

 other (P

lease state) 

  

 

F ‐ P

URC

HI FEES (give am

ount) 

Fee Deposite

d du

ring the current fin

ancial year till the

 last calenda

r mon

th: (Rs.) ________________________________ 

Purchi fee being charged @ Rs. ____________________________ per pa

tient.  

G ‐ PA

TIEN

TS TRE

ATED IN

 LAST CALENDAR MONTH

 (give nu

mbe

rs) 

Sr. N

o Ca

ses 

Num

bers 

Sr. N

o. 

Cases 

Num

bers 

1 OPD

 Cases 

 7 

Children vaccinated

 outside

 BHU 

 2 

Percen

tage of pe

rvious day 

OPD

 Ca

ses registered

 with

 NIC No.  

 8 

TB Patients un

der Treatm

ent 

 

3 Deliveries at BHU 

 9 

Hep

atitis “B” Va

ccination Don

e  

4 No. of P

CD slides prepared 

 10

 Anten

atal Cases Che

cked

  

5 No. of referrals to

 other hospitals 

 11

 Family Plann

ing Visits 

 6 

Children vaccinated

 at B

HU 

 12

 No. of referrals by LH

W 

  

H‐ SCH

OOL HEA

LTH PRO

GRA

M 

 S. No. 

Indicator 

Num

bers 

1 Total N

umbe

r of Stude

nts referred

 by SH

&NS du

ring

 previou

s mon

th 

 2 

Total N

umbe

rs of stude

nt treated at BHU referred by

 SH&NS 

 3. 

Total N

umbe

rs of Schoo

l visite

d du

ring

 previou

s mon

th 

 4. 

Tour program

 app

roved and displayed in DHQ 

Yes / No 

 I ‐ DOCT

ORS

 (give nu

mbe

rs) 

 

Sanction

ed Posts 

Filled Po

sts 

Presen

t  

  

This doctor is posted in th

is BHU but is not present during the visit  

Details reg

arding

 absen

ce of d

octors. (Do no

t write anything if a do

ctor is present.) 

Sr. N

o  

Designa

tion

 Nam

e of Doctors 

Type

 of A

bsen

ce on the mon

itoring da

y. 

(tick on

ly one

 box) 

Days of absen

ce during last 

three calend

ar m

onths 

UA 

SL 

OD 

St. L 

LC 

UA 

Other Types 

1 MO  W

MO 

  

  

  

  

 Una

utho

rized

 absence (U

A), Sanctio

ned leave (SL), O

n official duty ou

tside the BH

Y (OD), Short leave (St.L), Late Co

mer (LC).

 

J‐ PARA

MED

ICS (OTH

ER THAN DOCT

ORS

) 

  PARA

MED

ICS INCLUDES: 

Med

ical Assistant, M

edical Techn

ician, Health

 Techn

ician, Dispe

nser, O

ther 

  Sr. N

o Staff C

ategory 

Sanction

ed 

Filled Po

sts 

Presen

t 1 

Paramed

ical Staff 

  

 

Details reg

arding

 absen

ce of P

aram

edics. (D

o no

t write anything if staff is presen

t.) 

 

Sr. N

o  

Designa

tion

 Nam

e of Param

edic 

Type

 of A

bsen

ce on the mon

itoring da

y. 

(tick on

ly one

 box) 

Days of absen

ce during last 

three calend

ar m

onths 

UA 

SL 

OD 

St. L 

LC 

UA 

Other Types 

1  

  

2  

  

3  

  

 

K ‐ P

REVEN

TIVE / OUTR

EACH

 STA

FF 

  PREV

ENTIVE / OUTR

EACH

 STA

FF 

INCLUDES. 

LHV, RHI, Midwife

, Dai, Va

ccinator CDC Supe

rvisor, Sanitary Inspe

ctor Sanita

ry W

orker, 

Scho

ol Health

 & Nutritio

n Supe

rvisor 

 Sanction

ed Posts 

Filled Po

sts 

Presen

t  

  

 Details reg

arding

 absen

ce of P

reventive / ou

trea

ch staff. (Do no

t write anything if staff is presen

t.) 

 

Sr. N

o  

Designa

tion

 Nam

e of Staff 

Type

 of A

bsen

ce on the mon

itoring da

y. 

(tick on

ly one

 box) 

Days of absen

ce during last 

three calend

ar m

onths 

UA 

SL 

OD 

St. L 

LC 

UA 

Other Types 

1  

  

2  

  

3  

  

 

L ‐ A

DMIN / SUPP

ORT

 STA

FF 

  ADMIN / SUPP

ORT

 STA

FF IN

CLUDES: 

Compu

ter O

perator, Naib Qasid, Cho

wkidar, M

ali, Sw

eepe

r  

Sanction

ed Posts 

Filled Po

sts 

Presen

t  

  

 Details reg

arding

 absen

ce of A

dmin/Sup

port  staff. (Do no

t write anything if staff is presen

t.) 

 

Sr. N

o  

Designa

tion

 Nam

e of Staff 

Type

 of A

bsen

ce on the mon

itoring da

y. 

(tick on

ly one

 box) 

Days of absen

ce during last 

three calend

ar m

onths 

UA 

SL 

OD 

St. L 

LC 

UA 

Other Types 

1  

  

2  

  

3  

  

 

M‐ V

ACA

NT PO

STS (please write fu

ll na

me of posts) 

  Sr. N

o Nam

e of Post 

Num

ber of Vacan

t Po

st 

Sr. N

o. 

Nam

e of Post 

Num

ber of Vacan

t Po

sts 

  

 

  

 

  

 

  

 

  

 

31

N ‐ AVAILABILITY

 OF MED

ICINES (give nu

mbe

rs of tab

lets / bottles etc.) 

(Med

icines physically available on

 the da

te of visit in

 the stock & as pe

r Med

icines Stock Register) 

 

Sr. 

No. 

Med

icines 

Available 

Balance as on 

1st  of last 

mon

th (1

) 

Received

 Since 

1st  of last 

mon

th (2

) 

Total 

3=(1+2

) Co

nsum

ed 

since 1s

t  of last 

mon

th till 

toda

y  (4

) 

Balance as per 

register  

5=(3‐4)  

Yes 

No 

1 Ca

p. Amoxicillin 

  

  

  

 

2 Syp. Amoxicillin 

  

  

  

 

3 Tab. Cotrimoxazole 

  

  

  

 

4 Syp. Cotrimoxazole 

  

  

  

 

5 Any

 Other antibiotic

 Tablet 

  

  

  

 

6 Tab. M

etronidazole 

  

  

  

 

7 Syp. M

etronidazole 

  

  

  

 

8 Inj. Ampicillin 

  

  

  

 

9 Tab Diclofenac 

  

  

  

 

10 

Inj. Diclofenac 

  

  

  

 

11 

Syrup Paracetamol 

  

  

  

 

12 

Chloroqu

ine Tab 

  

  

  

 

13 

Syrup Salbutam

ol 

  

  

  

 

14 

Syp. Antihelminthic 

  

  

  

 

15 

I/V Infusion

s  

  

  

  

16 

Inj. Dexam

ethasone

  

  

  

  

17 

Iron

/Folic Tab. 

  

  

  

 

18 

ORS

 (Packets) 

  

  

  

 

19 

Oral Con

tracep

tive Pills 

  

  

  

 

20 

Anti‐H

istamine Tab.  

  

  

  

 

21 

Inj. Anti‐H

istamine 

  

  

  

 

22 

Anti‐T

uberculosis Drugs 

  

  

  

 

23 

Tetanu

s Toxoid Injections 

  

  

  

 

24 

Inj. Atrop

in 

  

  

  

 

25 

Inj. Adren

aline 

  

  

  

 

26 

Ant acid Tab. 

  

  

  

 

27 

Band

ages 

  

  

  

 

28 

Antisep

tic Solution (Bottle

s) 

  

  

  

 

29 

Dispo

sable Syringes 

  

  

  

 

 

O ‐ INSPECTION OF TH

E FA

CILITY

 BY DISTR

ICTGOVER

NMEN

T OFFICER

S From

 Inspection

 Reg

ister. (g

ive nu

mbe

r / da

tes) 

Sr. N

o. 

Inspecting

 Officer 

DDO (H

) DO (H

) ED

O (H

) DCO

 or his 

Represen

tative 

PRSP

 Re

presen

tative (in 

PRSP

 Dist. Only) 

1 Num

ber o

f inspe

ctions m

ade 

during

 the last six m

onths as 

per record of inspe

ction bo

ok 

  

  

2 Date of Last Inspe

ction 

  

  

 

 

P‐ PUBLIC OPINION (p

lease give num

ber of persons in

 the

 relevan

t colum

ns.) 

 

Views 

Num

ber of persons 

contacted in th

e catchm

ent area

 

PUBLIC OPINION 

Satisfactory 

Average 

Unsatisfactory

No Re

spon

se 

 1) Presence of Doctors 

  

  

 

2) Attitu

de of d

octors to

wards patients 

  

  

 

3) W

aitin

g Time 

  

  

 

4) Free availability of m

edicines 

  

  

 

5) Vaccinators outreach 

  

  

 

6) Vaccinatio

n at BHU 

  

  

 

  Note: Nam

es and

 Con

tact Num

bers of at least tw

o pe

rson

s interviewed

 during the visit 

  Sr. N

o. 

Nam

e Add

ress 

Contact N

umbe

r  

  

  

  

  

Q ‐ DEV

ELOPM

ENT SCHEM

ES / PRO

VISION OF MISSING FACILITIES (tick the column) 

  Sr. N

o Missing

 Facilities 

Fund

s provided

 by 

Status of W

ork 

Qua

lity 

Observation

s (use extra page 

if requ

ired

) PH

SRP 

District 

Gov

t. 

Not 

Started 

Halted 

% Com

pleted

 (give nu

mbe

r) 

Poor 

Avg. 

Goo

d 

1 BH

U Building  

  

  

  

  

 

2 Re

side

nces  

  

  

  

  

 

3 Bo

undary wall 

  

  

  

  

 

4 Electricity

  

  

  

  

  

5 Drinking Water 

  

  

  

  

 

6 Latrine/Toilet 

  

  

  

  

 

7 Furnitu

re Sui Gas 

  

  

  

  

 

8 Sewerage 

  

  

  

  

 

9 Other 

  

  

  

  

 

 

R (i)‐ M

EDICAL EQ

UIPMEN

T (give nu

mbe

rs) 

  Sr. 

No. 

Nam

e of Item

 Available 

Function

al 

If Non

‐Fun

ctiona

l Re

marks 

Repa

irab

le 

Unserviceab

le 

1 Delivery Table 

  

  

 

2 Delivery Light 

  

  

 

3 Hospital Bed

s  

  

  

4 Sucker 

  

  

 

5 Oxygen Cylinde

rs 

  

  

 

6 Autoclave 

  

  

 

7 Glucometer 

  

  

 

8 Safe Delivery Kit  

  

  

 

9 Em

ergency tray with

 life saving med

icines 

  

  

 

10 

Ambu

 Bag 

  

  

 

11 

Bulb Sucker  

  

  

 

12 

Baby

 Warmer 

  

  

 

 R (ii) –

 NON‐M

EDICAL EQ

UIPMEN

T (give nu

mbe

rs) 

  Sr. N

o. 

Nam

e of Item

 Available 

Function

al 

If Non

‐Fun

ctiona

l Re

marks 

Repa

irab

le 

Unserviceab

le 

1 Co

mpu

ter 

  

  

 

2 Printer 

  

  

 

3 UPS

  

  

  

 S – RE

SIDEN

CES (give nu

mbe

rs) 

  Sr. N

o. 

Nam

e of Post 

Reside

nce 

Available 

Reside

nce 

Occup

ied 

(Yes/N

o) 

Physical Status of Residen

ce 

Remarks 

Reside

 able 

Not reside ab

le 

1  

  

  

 

2  

  

  

 

3  

  

  

 

 T ‐ SER

VICES 

  Sr. N

o. 

 YES 

NO 

 AIDS & HEP

ATITIS CO

NTR

OL 

  

1 Syringe cutters available  

  

2 Syringe cutters be

ing used

  

  

EPI 

  

3 Co

ld Chain intact 

  

4 All vaccines available at EPI Cen

ter 

  

5 NATIONAL PR

OGRA

M FOR FP

 & PHC 

  

 LH

W m

onthly m

eetin

g he

ld 

  

6 LH

W m

onthly m

eetin

g repo

rt com

pleted

  

 7 

Mon

thly sup

plies / med

icines rep

lenished

   

 8 

MCH

  

  

Labo

ur Roo

m equ

ipmen

t available 

  

9 Labo

ur Roo

m equ

ipmen

t fun

ctional  

  

10 

Family plann

ing services being

 provide

d  

 11

 LO

GISTICS

  

 12

 Gen

eral sup

plies available (Linen

s, bed

side

 lockers, etc.) 

  

32

       U ‐ MONTH

LY PER

FORM

ANCE

   Sr. N

o  

Mon

thly Target 

Performan

ce 

1 No of children given full im

mun

ization coverage 

  

2 Daily OPD

 atten

dance 

  

3 Delivery coverage at facility 

  

   

V– GEN

ERAL RE

MARK

S          

Time of Dep

arture from

 the facility: Hou

rs ________ Minutes ________      am      /      pm 

    Certified

 that th

is basic Hea

lth Unit was inspected toda

y by

 the un

dersigne

d an

d the inform

ation stated

 abo

ve is as 

per facts an

d record. 

         ______________

                    

   ________________________________

                                _________________

 Signature of DDOH/M

EA                Signa

tures & Stamp of M

O/Incha

rge                                          Signa

tures of DMO/EDOH 

33

C Training Manual For Smartphone Application Use

Manual

for

Health Facility Information

Aggregation System

Directorate General Health Services

Supported By

Punjab Health Sector Reforms Program

Directorate General Health Services, Health Department, Government of the Punjab

CONTENTS

1. Introduction ..................................................... 1

2. About the phone .............................................. 3

3. About the application ...................................... 4

3.1. How to update forms if notified ........................ 7

3.2. How to fill a form ............................................ 10

3.2.1. How to fill a BHU form ........................... 11

3.2.2. How to fill an RHC form ......................... 20

3.2.3. How to fill a THQ form ........................... 30

3.2.4. How to fill a DHQ form ........................... 39

3.3. How to submit completed forms ..................... 48

Directorate General Health Services, Health Department, Government of the Punjab

1

1. INTRODUCTION

The Health Department of Government of

the Punjab is committed to adopting state-

of-the-art technology to strengthen

governance and improve service delivery

for all citizens.

For this purpose, the Punjab Health Sector

Reforms Program (PHSRP), with

technical assistance from International

Growth Centre (IGC) Team, is supporting

DGHS and district health managers in

strengthening the internal monitoring

system of the Health Department. This is

being done by introducing a mobile phone

based information management system

that is being rolled out across different

districts of the province.

This initiative will improve the internal

information transmission within the

Health Department and will ensure that

timely, authentic and actionable

information is sent quickly from

individual facilities to district and

provincial health managers on such

crucially important issues as absenteeism,

medicine stock outs, availability and

functionality of equipment etc.

Android-based smartphones have been

provided to those district supervisory

officers, such as Executive District Health

Officers (EDOs), District Health Officers

(DOs), and Deputy District Health

Officers (DDOs), who have been tasked

with the collection of performance related

data from Basic Health Units (BHUs),

Rural Health Centers (RHCs) and Tehsil

and District Headquarters (THQs and

DHQs).

The report submitted by these officers

through the phone will be recorded on a

website and automatically analyzed for

use by managers at various levels. It is

expected that this information will

become a powerful tool for management

both for district and central level officials.

This is expected to bring about marked

improvement in health service delivery

management, particularly at primary and

secondary levels of healthcare, leading to

better health outcomes for the poor and

disadvantaged in the province.

At Directorate General Health Services,

Director, District Health Information

System (DHIS), supported by the PHSRP

and IGC team, is the focal person for

implementation of the program at the

provincial level. Overall responsibility for

the program at the district level lies with

EDOs, and Statistical Officers (SOs) are

the designated focal persons for managing

the system at the district level.

This manual contains basic information

about the program and the phone, as well

Directorate General Health Services, Health Department, Government of the Punjab

2

as details of how to submit data and deal

with some problems that may arise.

34

Directorate General Health Services, Health Department, Government of the Punjab

3

2. ABOUT THE

PHONE

The HTC Explorer runs on Android 2.3.5

with HTC’s latest custom interface -

Sense 3.0, and is equipped with a 3.2 inch

capacitive touch screen.

The phone has 4 capacitive touch buttons

on the front- HOME, MENU, BACK and

SEARCH.

With a 600 MHz processor based on the

latest mobile technology, 512 MB of

RAM and a 2 GB SD card, the phone is

well equipped to deal with advanced

tasks associated with smart-phones

today.

The phone can be used for

browsing the internet using either

GPRS or WIFI. It is also equipped

with a GPS device and a 3 MP

camera which can capture high-

resolution images and videos.

For detailed instructions regarding

how to undertake different tasks

on the phone and a comprehensive

guide to unlocking the full

potential of the device, please visit

the following website:

http://www.htc.com/uk/help/htc-

explorer/#overview

If you encounter any further

problems while using the phone,

please contact the helpline given

at the end of this document.

Directorate General Health Services, Health Department, Government of the Punjab

4

3. ABOUT THE

APPLICATION

The Android application is very intuitive

and simple to use. Before running the

application, you must ensure that you are

connected to the internet and the GPS is

switched on. To confirm that you are

connected to the internet, tap the

‘Internet’ icon on the home screen to

launch the phone browser and try opening

any webpage (e.g. yahoo.com); if the

webpage opens up, it means you are

connected to the internet. In this case, tap

the phone’s ‘HOME’ capacitive touch

button to return to the home screen. To

confirm if GPRS (internet) is enabled or

not, tap the phone’s ‘MENU’ capacitive

touch button while on the home screen

and select ‘Settings’ tab that pops on the

bottom right of the screen, as shown

below:

Choose ‘Wireless & networks’ from the

list of settings that appear on the screen.

Then scroll down the page to check

whether the option of ‘Mobile network’ is

selected or not.

Directorate General Health Services, Health Department, Government of the Punjab

5

If it is selected, as shown, then the GPRS

is switched on. If not, switch it on by

checking this option. Confirm again by

returning to the home screen by tapping

“HOME” and opening any webpage using

the phone’s browser. If it still does not

open, report the issue on the helpline

given at the bottom of this document. If

the website opens, go back to the home

screen.

To check if the GPS is on or off, check

the power control widget on the main

screen (the dark grey bar at the top with

five large symbols); if the GPS symbol is

highlighted, as shown below, the GPS is

on. If not, tap the GPS symbol to toggle it

on, before starting the application.

Once it is confirmed that the phone is

connected to the internet and the GPS is

switched on, tap the PHSRP icon on the

home screen to start the application.

The application main screen has three

buttons- ‘Start New Form’, ‘Send

Finished Forms’ and ‘Manage

Application’- as shown below:

Directorate General Health Services, Health Department, Government of the Punjab

6

In order to start making entries, the

application needs to first download the

relevant forms. There are four forms for

each district; one for each type of facility-

BHU, RHC, THQ and DHQ. For the case

of the phones handed out, the relevant

forms have already been downloaded.

However, in case there are any revisions

made, all concerned officials will be

notified that the forms will have to be

updated. Do not delete the forms unless

you are formally notified to do so.

35

Directorate General Health Services, Health Department, Government of the Punjab

7

3.1. How to update

forms if notified

To update the forms on the application if

you are notified to do so, tap the

‘Manage Application’ button.

Considering Pakpattan as an example, the

following screen will be displayed:

Select all the forms, or just the ones that

you need to update as notified, by tapping

on the checkboxes on the right, and tap

the ‘Delete Selected’ button at the bottom

right. A confirmation will be displayed as

follows:

Tap ‘Delete Items’ to confirm and the

selected forms will be deleted. If all the

forms are deleted, the following screen

will be displayed:

Now, tap ‘Get New Forms’, to retrieve

the updated forms. The application will

use the internet to list the updated forms

of all districts for download as follows:

Directorate General Health Services, Health Department, Government of the Punjab

8

If you encounter an error at this point, it

means you are not connected to the

internet. Ensure that you are connected to

the internet by following the instructions

given previously and try again. If you

encounter an error again, report the issue

immediately on the helpline given at the

end of this document. If there is no error

and the above screen is displayed, scroll

vertically to find the forms of your district

and select them all by tapping the

checkboxes on their right as shown:

Then, tap ‘Get Selected’ to download the

updated forms of your district. Once the

forms are successfully downloaded, the

following screen will be displayed:

If there is some sort of error at this point,

try downloading the forms again. If you

are still unsuccessful, report the issue on

the helpline to get an immediate solution.

Directorate General Health Services, Health Department, Government of the Punjab

9

If all forms are successfully downloaded

and the above screen is displayed, tap

‘OK’ and the following screen will be

displayed:

Tap the phone’s ‘BACK’ capacitive touch

button at the bottom of the screen to get to

the main screen of the application again.

You are all set to continue to making and

submitting entries now.

Directorate General Health Services, Health Department, Government of the Punjab

10

3.2. How to fill a

form

At this point, it is important to note that

completing a form and submitting a form

are two different tasks that are performed

separately. Filling a form does not require

an internet connection, so you can enter

data from your inspection visits and save

the completed forms regardless of

whether the internet is working or not.

However, submitting the forms requires

an internet connection.

To start filling a form, tap the ‘Start New

Form’ button from the main screen of the

application.

The following screen will be displayed,

prompting you to choose the type of

facility:

Before moving on, it is important to note

that if you want to close or discard the

entry at any point before saving and

exiting, tap the BACK capacitive button

on the phone and choose ‘discard entry’.

If you tap BACK by mistake, simply tap

‘Cancel’ on the dialogue box that pops

up.

Furthermore, if you accidentally tap the

phone’s ‘HOME’ capacitive touch button

and end up at the home screen while

filling in the form, simply tap the PHSRP

application icon again to load the

application again and it will return you to

the screen you were previously at in the

form with all previous entries made on the

form intact.

36

Directorate General Health Services, Health Department, Government of the Punjab

11

3.2.1. How to fill a

BHU form

To fill a BHU form, choose the BHU

form from the list shown above and the

following screen will be displayed,

instructing how to navigate through the

form:

It is important to note here that you will

be able to scroll back and forth within the

form to check or change your entries

before you complete the form, by

scrolling laterally in one direction or the

other, but whenever you scroll to a screen

that requires numerical input from the

keypad that pops up (as explained later),

all numerical entries will be cleared and

you will have to re-enter them.

Scroll laterally, as instructed, to start

filling in the form. The next screen will

allow you to choose the Tehsil in which

the BHU is located, as shown below:

It is important to note at this time that

some screens require at least one entry by

the user, and you will not be able to move

forward in the form unless it is made. To

demonstrate, if you attempt to move

forward in the form by scrolling laterally

when it prompts you to enter the Tehsil in

which the facility is located, the following

message will appear on the screen:

Directorate General Health Services, Health Department, Government of the Punjab

12

You will have to select one of the options

and then scroll laterally to move to the

next screen. The next screen will require

you to choose the BHU you are visiting

from a list of all the BHUs present in that

Tehsil. For demonstration, we select the

Tehsil of Arifwala and scroll to the next

screen. The following list is displayed:

Scroll vertically to find and choose the

specific facility you are visiting, and

scroll laterally to move to the next screen:

This screen relates to the availability

status of the Medical Officer at the

facility. An important thing to note here is

that for all non-PRSP districts, the last

option will not be shown on this screen as

it does not apply to them. As Pakpattan is

a PRSP district, the ‘Gone to other BHU’

option is available on the form.

Another important thing to note here is

that all officers are required to make these

entries from the perspective of a citizen

visiting the facility- so even if the MO is

on official leave or out on some official

business at the time of the inspection

visit, he/she would be marked absent.

However, officers would also be required

to take a note regarding the reason for

absence of the MO in their diaries for

such exceptional cases.

Directorate General Health Services, Health Department, Government of the Punjab

13

For demonstration, we choose Present and

scroll laterally to the next screen:

This screen requires you to check all the

people not present at the BHU. As

mentioned for the case of the MO, the

officer will mark people absent based on

the perspective of a visiting citizen- if

someone is out on official business or on

official leave or even if the position is not

filled etc., the position holder will be

marked as absent, and a note will be made

in the officer’s diary about the reason for

absence for these exceptional cases.

If all the staff is present, you can scroll

laterally to move to the next screen

without marking any checkbox on this

screen. The next screen requires you to

mark tablets not available at the facility,

as shown:

Scroll vertically and mark all the tablets

that are out of stock at the BHU. If all

tablets are present, scroll laterally to the

next screen without marking any

checkbox.

Repeat the same procedure for

‘Injections’, ‘Syrups’ and ‘Other

Medicines’ in the subsequent screens as

shown:

Directorate General Health Services, Health Department, Government of the Punjab

14

The next screen will require you to mark

all equipment that is not functional.

Unavailable equipment will also be

marked as non-functional:

Leave the screen unmarked if all

equipment is available and functional, and

scroll laterally to the next screen.

37

Directorate General Health Services, Health Department, Government of the Punjab

15

The next screen will require you to tap in

numerical values for the number of OPD

cases last month, number of deliveries last

month and number of Antenatal cases last

month. A keypad will pop up at the

bottom automatically so that you can

enter the numbers. Tap on the entry bar of

the next field to enter its number after you

are done with the first one, and then move

on to the third one after you are done with

the second one. All three fields must be

filled in order to move to the next screen.

To get to the third field, you will have to

scroll vertically lower down the page.

While scrolling, ensure that you are

avoiding the keypad, as scrolling over the

keypad will not work.

Once all three entries are filled, scroll

laterally to move to the next screen. Once

again, ensure that you avoid the keypad as

scrolling laterally over the keypad will not

work.

The next screen will require you to enter

the mobile numbers of any two

randomly selected delivery patients from

the BHU records from last month. The

entry fields are designed to detect invalid

numbers, and the application will not let

you move to the next screen unless you

enter two valid mobile numbers.

Once the two mobile numbers are entered,

scroll laterally to move to the next screen.

The next screen will require you to enter

mobile numbers of any two randomly

selected ANC patients from last month.

The entry fields on this page are also

designed to detect invalid numbers, and

the application will not let you move to

the next screen unless you enter two valid

mobile numbers.

Directorate General Health Services, Health Department, Government of the Punjab

16

Once the numbers are entered, scroll

laterally to move to the next screen:

Choose the most appropriate option and

scroll laterally to move to the next screen.

This screen will require you to mark

which information was displayed in the

BHU. Leave the screen unmarked and

scroll laterally to the next screen if none

of these were displayed at the facility.

Mark the options appropriately and scroll

laterally to move to the next screen.

Directorate General Health Services, Health Department, Government of the Punjab

17

The next screen will require you to take a

clear picture of yourself with the essential

staff present at the BHU, as shown below:

Tap the ‘Take Picture’ button to load the

camera. For better picture quality, it is

advisable to take the picture indoors and

have someone take it for you. To take the

picture, have that person tap the silver

button in the centre-bottom of the screen,

as shown below:

When the picture is taken, you will be

given the option of retaking it if you are

not satisfied with it. Tap the camera icon

on the right to load the camera again and

take a better picture, as shown below:

Once you are satisfied with the picture,

tap the ‘Done’ button on the left, and you

will be taken to the following screen:

Directorate General Health Services, Health Department, Government of the Punjab

18

Scroll laterally to move forward. If,

instead, you want to view the picture in

full screen again, tap the picture preview

box at the bottom, and you will be able to

view it in full screen:

Tap the phone’s ‘BACK’ capacitive touch

button at the bottom to return to the

previous screen. Once there, scroll

laterally again to move to the next screen:

Tap ‘Record Location’ and the phone will

record its location using GPS, network

information and GPRS. It is advisable to

move outdoors to record location as GPS

signals are stronger outdoors. While you

wait for the location to be recorded, you

might see the accuracy radius values

decreasing gradually:

38

Directorate General Health Services, Health Department, Government of the Punjab

19

When accuracy radius falls to 5 m, the

following screen will be displayed:

GPS satellites are not always in range

hence it might take some time for the

phone to narrow down its location. If,

even after waiting for five to ten minutes,

the phone is unable to record its location,

ensure that the GPS is toggled on and try

again. If, still, the phone is unable to

record its location, contact the helpline

immediately for quick resolution. Once

the location is recorded, the above screen

will be displayed. To move forward,

scroll laterally again to get to the

following screen:

Tap ‘Save Form and Exit’ to complete the

entry. A message will be displayed

notifying you that the form was saved

successfully and you will be taken back to

the main screen of the application.

Directorate General Health Services, Health Department, Government of the Punjab

48

3.3. How to submit

completed forms

Once you have completed the form (after

pressing the ‘Save Form and Exit’

button), it needs to be submitted. After

completing the form, tap the ‘Send

Finished Forms’ button on the application

main screen:

This will take you to a screen where all

your completed and un-submitted forms

are listed. Select the one you would like

to submit or select all if you want to

submit all, and tap the ‘Send Selected’

button on the bottom right of the screen.

If the submission was successful, a

message will appear saying so, and the

respective completed forms will vanish

from this list. If all were selected and

successfully sent, all will disappear. Tap

the phone’s ‘BACK’ capacitive button to

return to the application’s main screen.

If there is any error in submission, it can

be because of the internet not working. In

that case, confirm if the internet is

working and try submitting the form/s

again. If you are still unsuccessful, report

your issue on the helpline given at the end

of this document.

Tap the phone’s ‘HOME’ capacitive

touch button to exit the application and

return to the home screen of the phone

once you have successfully submitted the

forms.

Helpline: 0308 4091080

39

D Training Manuals For Dashboard Use

Manual

For

Health Facility Information

Management System

(Online Dashboard)

Directorate General Health Services

Supported By

Punjab Health Sector Reforms Program (PHSRP) International Growth Centre (IGC)

Lahore University of Management Sciences (LUMS)

Directorate General Health Services, Health Department, Government of the Punjab

CONTENTS

1. Introduction .......................................................................... 1

2. The Dashboard ..................................................................... 3

2.1. The District Level ...................................................... 5

2.1.1. Compliance Status ................................................. 5

2.1.2. Facility Status ........................................................ 8

2.1.3. Recent Visits ........................................................ 11

2.1.4. Indicators ............................................................. 12

2.1.5. Time Trend Charts ............................................... 14

2.1.6. Photo Verification ............................................... 16

2.1.7. Map ...................................................................... 17

2.2. The Provincial Level ............................................... 20

2.2.1. Compliance Status ............................................... 20

2.2.2. Facility Status ...................................................... 22

2.2.3. Indicators ............................................................. 23

2.2.4. Time Trend Charts ............................................... 24

Appendix ................................................................................. 26

Directorate General Health Services, Health Department, Government of the Punjab

1

1. Introduction

The Health Department of Government of the Punjab is committed to

adopting state-of-the-art technology to strengthen governance and improve

service delivery for all citizens.

For this purpose, the Punjab Health Sector Reforms Program (PHSRP), with

technical assistance from International Growth Centre (IGC) Team, is

supporting DGHS and district health managers in strengthening the internal

monitoring system of the Health Department. This is being done by

introducing a mobile phone based, online information management system.

This initiative will improve the internal information transmission within the

Health Department and will ensure that timely, authentic and actionable

information is sent quickly from individual facilities to district and provincial

health managers on such crucially important issues as absenteeism, medicine

stock outs, availability and functionality of equipment etc.

Android-based smartphones have been provided to those district supervisory

officers, such as Executive District Health Officers (EDOs), District Health

Officers (DOs), and Deputy District Health Officers (DDOs), who have been

tasked with the collection of performance related data from Basic Health

Units (BHUs), Rural Health Centers (RHCs) and Tehsil and District

Headquarters (THQs and DHQs).

The report submitted by these officers through the phone will be recorded on

a website, known as the ‘Dashboard’, and automatically analyzed for use by

managers at various levels. It is expected that this information will become a

powerful tool for management both for district and central level officials.

This is expected to bring about marked improvement in health service

delivery management, particularly at primary and secondary levels of

healthcare, leading to better health outcomes for the poor and disadvantaged

in the province.

Directorate General Health Services, Health Department, Government of the Punjab

2

At Directorate General Health Services, Director, District Health Information

System (DHIS), supported by the PHSRP and IGC team, is the focal person

for implementation of the program at the provincial level. Overall

responsibility for the program at the district level lies with EDOs, and

Statistical Officers (SOs) are the designated focal persons for managing the

system at the district level.

This manual explains what information is available on the online dashboard

and how it is displayed, to help managers at different levels to utilize this

powerful tool to its full potential in order to improve health care in the

province.

40

Directorate General Health Services, Health Department, Government of the Punjab

3

2. The Dashboard

The online dashboard can be accessed any time over the internet through the

following link:

punjabmodel.gov.pk/phsrp/dashboard

When you open the link, the following page will be displayed, prompting

you to enter your username and password, and giving you the option of

saving these credentials for automatic login the next time you open the link,

as shown in Figure 1.

Figure 1

To access the dashboard, you have to enter the unique username and

password already communicated to you and click on ‘Login’. Once

successfully logged in, you can also change your password for the dashboard

by accessing the Change Password section in the blue bar. When you are

done using the dashboard, you can click on ‘Logout’ to end the session.

Figure 2

Directorate General Health Services, Health Department, Government of the Punjab

4

As shown in Figure 2, the blue bar near the top of the page contains all the

major sections of the dashboard, allowing you to effortlessly navigate from

one part of the online tool to another.

One major feature of this tool is the ‘Print’ button/icon which is located to

the right, just below the blue bar. Clicking this allows you to take a snapshot

of whatever is currently being displayed on the dashboard and print it out.

It is important to note that there are two levels of access for the dashboard-

the district level and the provincial level. All DCOs, EDOs, DOs and DDOs

have access to the district level but not the provincial level, ergo when they

log in, they are shown the district level by default. The relevant higher up

senior officers, however, have access to the district level as well as the

provincial level, so when they log in, their default view is the provincial

level, but they can also choose to access the district level by choosing from a

drop down list of districts near the top of the webpage.

Directorate General Health Services, Health Department, Government of the Punjab

5

2.1. The District Level

2.1.1. Compliance Status

The first page that is displayed when you log in the dashboard is the

Compliance Status section. Officers can use this section to track their

compliance performance for the current month as well the months before.

They can also gauge their current standing compared to fellow health

officers in the district with respect to compliance.

The most prominent characteristics of this page are the 2 bar charts and the

table below them.

The first bar chart represents the percentage compliance of all the health

officers in the district for the last calendar month, disaggregated by facility

type. This is calculated as follows:

Percentage compliance= (total visits performed last month / visits assigned

last month) x 100

The bars are color coded by facility type, as explained by the legend

displayed on the page. Compliance is 100% if the officer performed 100% of

the visits assigned to him or more.

The second bar chart represents the percentage coverage of all health officers

in the district for the last calendar month, disaggregated by facility type.

This is calculated as follows:

Percentage coverage= (1 – (no. of assigned facilities not visited by any

officer last month/ facility count)) x 100

Once again, the bars are color coded by facility type, as explained by the

legend displayed on the page, as shown in Figure 3.

Directorate General Health Services, Health Department, Government of the Punjab

6

Figure 3

41

Directorate General Health Services, Health Department, Government of the Punjab

7

The distinction between what the two charts convey is important and is

easily explainable using an example. Suppose there are 10 facilities in an

officer’s jurisdiction and he is assigned a total of 10 visits. If he visits every

single facility once, his compliance as well as his coverage will be 100%. If

he visits only 1 facility 10 times during the month, his compliance will still

be 100% but his coverage will be 10%. Similarly, suppose if the assigned

visits are 20 and the facility count is still 10; if he visits each facility once

(leading to a total of 10 visits), his compliance will be 50% but his coverage

will be 100%. Officers should strive for 100% compliance as well as the

maximum possible coverage (which can be less than 100% only in cases

where facility count exceeds the number of assigned visits).

The table below the charts gives detailed information regarding compliance

figures. The ‘+’ icon before every officer’s designation in the ‘Supervisory

Officer’ column can be clicked to expand the table to show information

disaggregated by facility type. The information displayed in the table

includes the facility count, monthly assigned visits, unique and total visits

performed during the current month, unique and total visits performed last

month, and the percentage compliance for last month, for every officer in the

district, disaggregated by facility type as well as in total.

For cases in which compliance in the last calendar month is low, the table is

highlighted red, as shown in Figure 4.

Figure 4

The last column provides hyper links, allowing you to jump directly to the

relevant entries in the ‘Recent Visits’ section. The Recent Visits section will

be explained in detail later on.

Directorate General Health Services, Health Department, Government of the Punjab

8

If you are interested to see compliance figures for months before the last

calendar month, you can click on the ‘View Detailed Report’ hyperlinked

text located near the top of the page.

Note: Should you find that a visit to a particular facility is not being

displayed on the dashboard despite being successfully submitted from the

Android smart phone allotted to you, please convey it immediately at the

helpline given at the end of this document.

2.1.2. Facility Status

The Facility Status section gives you a list of all the facilities in the district,

arranged by the date of last visit with the oldest visited at the top. It is

designed to enable you to keep track of facilities that are being neglected.

The facilities are color coded, according to the legend displayed on the page,

as shown in Figure 5.

Figure 5

Directorate General Health Services, Health Department, Government of the Punjab

9

The page has different tabs for the different facility types. Each tab displays

a table which displays the facility name, the Tehsil/Town it is located in, the

designation of the officer who last visited the facility, the date of the last visit

and the number of days since the last visit. The corresponding columns also

have filters in-built that allow you to view selective information if you

choose to.

The table also contains a column for Summary Report. Clicking the icon in

this column for any row will take you to a page displaying details regarding

the last visit to the facility as well as the second last visit, in addition to

Tehsil variable averages (from 30 days from the last visit). Figure 6 shows a

cropped screenshot of the page.

Figure 6

Directorate General Health Services, Health Department, Government of the Punjab

10

Clicking on the icon in the Recent Visits column for any facility, instead,

will take you to the Recent Visits section showing you a list of all entries

made for that facility, as shown in Figure 7.

Figure 7

Officers should ensure that all the facilities listed in the Facility Status

section are green- some can be blue for cases in which the facility count is

more than the assigned visits. Orange or red rows represent neglected

facilities and they should be visited as soon as possible.

42

Directorate General Health Services, Health Department, Government of the Punjab

11

2.1.3. Recent Visits

The Recent Visits section lists all entries as they come in, with the latest

submitted on top. There are different tabs for different facility types. Each

facility type tab contains a date filter, which allows you to view entries

submitted during a particular time period, and a table consisting of entries, as

shown:

Figure 8

To view entries submitted between certain dates, choose the start and end

dates from the drop down calendars displayed by clicking on the two white

text boxes immediately below the facility type tabs respectively, and click

the ‘Filter by Period’ button.

Directorate General Health Services, Health Department, Government of the Punjab

12

Some of the entries in the table might be highlighted red, as shown in the

above screenshot. These represent facilities where significant staff absence

was reported. The table also allows you to display only the highlighted

entries or the non-highlighted entries separately, in addition to displaying

them all together. The drop down filter for the column labeled ‘Absence’ can

be used to toggle between the selections.

The table also contains information that includes the facility name, the

Tehsil/Town it is located in, the visiting officer, the date of visit, the

availability status of the MO and the availability status of other staff. It also

provides filters for all these categories for selective searches.

The Summary Report icon at the end of every entry in the table can lead you

to a page displaying details regarding the last visit to the facility as well as

the second last visit, in addition to Tehsil/Town variable averages (from 30

days from the last visit) as already depicted in Figure 6.

As already mentioned, should you find that a visit to a particular facility is

not being displayed on the dashboard despite being successfully submitted

from the Android smart phone allotted to you, please convey it immediately

at the helpline given at the end of this document.

2.1.4. Indicators

The Indicators section displays charts comparing performance of the

different Tehsils/Towns based on the various indicators reported during

facility visits. Once again, there are different tabs for different facility types,

and different indicators, in some cases, for different tabs. The following

screenshot should give you an idea of what the page looks like:

Directorate General Health Services, Health Department, Government of the Punjab

13

Figure 9

It is important to note that while there are multiple BHUs, RHCs and THQs

in each district, the number of DHQs is one or zero. Hence, instead of a

comparison across Tehsils/Towns as for the case of BHUs, RHCs and THQs,

the DHQ section compares DHQs across districts. Furthermore, all

indicator charts that display data expressed in percentages in the DHQ

section have an additional red bar which reflects percentage compliance in

every district. The compliance bars are intended to be a gauge of how many

visits’ data is used to derive the charts- ergo, the higher the compliance, the

more reflective is the value of the variable of the actual situation in the

corresponding district.

For all tabs, there is a text box allowing you to choose which month you

want to see the data for. The page displays charts for the last calendar month

by default. If you want to access charts for some previous month, you need

to click on the white text box, select the month and year from the drop down

Directorate General Health Services, Health Department, Government of the Punjab

14

menu, click ‘Done’, and then click the ‘Update’ button located to the

immediate right.

Most indicators in the list have multiple charts that are displayed when you

click on any one of them. All charts have descriptive labels that clearly

indicate what they represent. Tables 1 through 4 in the appendix show how

the charts are arranged for each facility type.

These charts can prove to be a very powerful tool for Tehsil-wise

comparison based on the different performance related indicators. However,

if taken in isolation, interpretations derived from them may be misleading.

For example, if Tehsil ‘A’ shows 0% MO absence while Tehsil ‘B’ shows

20% MO absence, it doesn’t necessarily imply that Tehsil ‘A’ is better in

MO attendance than Tehsil ‘B’. It is possible that only a single visit was

performed in Tehsil ‘A’ in the entire month- during which the MO was

present- while, out of the 10 visits performed in Tehsil ‘B’, the MO was

absent in only 2. Ergo, the information displayed in the charts should always

be interpreted while considering compliance figures.

2.1.5. Time Trend Charts

The Time Trend Charts section contains line graphs representing the change

over time in all the indicators of the different facility types present in the

Indicators section as shown in Tables 1, 2, 3 and 4 in the appendix. The

general layout of this section is very similar to that of the Indicators section,

with the same indicator tabs and option to select a different month for all

facility types. However, there is one key difference; the charts contain two

lines- a thin one representing the district average and a thick one representing

the provincial average- allowing you to compare the average district

performance on each indicator to the provincial average, over time, instead

of comparing across Tehsils/Towns of the same district. Figure 10 shows

how the webpage might look.

43

Directorate General Health Services, Health Department, Government of the Punjab

15

Figure 10

You can also compare the performance of any Tehsil/Town compared to the

district average over time. This can be done by clicking the drop down

button near the top of the page and selecting the Tehsil/Town you want to

compare with the district average. In the charts that will be displayed as a

result, the thick line would represent the district’s average and the thin line

would represent the Tehsil/Town average.

These charts can prove to be very useful in observing and comparing trends

in different indicators over time, at the provincial, district, as well as the

Tehsil/Town level.

Directorate General Health Services, Health Department, Government of the Punjab

16

2.1.6. Photo Verification

To verify staff presence, the smart-phone Performa requires officers to take

pictures of the essential staff present at the facility they are visiting. The

Photo Verification section displays all these, sorted by the most recent visit,

by officer designation. Figure 11 shows the layout of the page.

Figure 11

You can view the full size version of any picture by clicking on it. Health

officers responsible for supervision of BHUs, RHCs, THQs and DHQs are

advised that the pictures submitted should not be blurry or unclear in any

way for the convenience and effectiveness of photo verification.

Directorate General Health Services, Health Department, Government of the Punjab

17

2.1.7. Map

When you click on the tab for the Map section, a separate window (or tab,

depending on your browser) will open, displaying a map of Pakistan and its

surrounding areas as shown in Figure 12.

Figure 12

For completing an entry for a facility visit, the smart-phone Performa

requires the supervisory officer to record the location of the facility using the

phone’s GPS. All successfully submitted entries show up on this map when

you zoom down to individual district.

In order to view entries for any district, you need to click on the relevant

district tab from the list on the left. Once you zoom in, all the relevant entries

will show up as place-marks color-coded with respect to the facility type, as

shown in Figure 13.

Directorate General Health Services, Health Department, Government of the Punjab

18

Figure 13

You can zoom further in or out using the zooming tool in the upper left

corner of the map. The map also allows you to show or hide District and

Tehsil boundaries, and even switch between Map and Satellite view.

Furthermore, the date filter allows you to see only those entries submitted

during a certain time period.

Clicking on any place-mark reveals a few details regarding the entry that

include the supervisory officer’s designation, the date the entry was made,

the start and end time of the visit and a link to the picture taken for the entry,

as shown in Figure 14.

44

Directorate General Health Services, Health Department, Government of the Punjab

19

Figure 14

The map allows for spatial review of the coverage and compliance in the

District or Tehsil/Town, which can prove to be very useful for circumstances

in which information regarding the location and spread of the facilities is

crucial.

Directorate General Health Services, Health Department, Government of the Punjab

20

2.2. The Provincial Level

As already mentioned, when you log in to the dashboard with an account that

has provincial level access as well as district level access, your default view

of the dashboard is the provincial level view. However you can access the

district level view for any district by choosing it from the drop down list that

appears when you click the ‘Punjab’ button, which is right below the blue

bar near the top of the page.

The Recent Visits and Photo Verification sections in the provincial level

view are blank as the usefulness of a combined list of entries or verification

pictures coming in from all districts is very limited.

Apart from that, the Map section for both the levels is exactly the same.

2.2.1. Compliance Status

Once again, the first page displayed after a successful login is the

Compliance Status section. This is just like the Compliance Status section in

the district level view except that instead of a comparison across

Tehsils/Towns in a district, you have a comparison of compliance across

districts.

The bars in the two charts are color-coded in the same way as in the district

level view, and the table below the charts gives detailed information

regarding compliance figures for districts, rather than supervisory officer.

Again, the ‘+’ icon can be clicked to expand the table to show information

disaggregated by facility type. The information displayed in the table

includes the facility count, monthly assigned visits, unique and total visits

performed during the current month, unique and total visits performed last

month, and the percentage compliance for last month, for every district,

disaggregated by facility type as well as in total.

Figure 15 shows how the page might look like.

Directorate General Health Services, Health Department, Government of the Punjab

21

Figure 15

Directorate General Health Services, Health Department, Government of the Punjab

22

Districts with low compliance in the last calendar month will be highlighted

in red. The last column provides hyperlinks, allowing you to jump directly to

the relevant entries in the ‘Recent Visits’ section, as in the district level view.

Moreover, if you are interested to see compliance figures for months before

the last calendar month, you can click on the ‘View Detailed Report’

hyperlinked text located near the top of the page, in same way.

This section is very useful for senior officials to track the compliance and

coverage status of all districts and compare them if need be.

2.2.2. Facility Status

The Facility Status section in the provincial level view is radically different

from that in the district level view, as apparent from Figure 16.

Figure 16

45

Directorate General Health Services, Health Department, Government of the Punjab

23

The page displays a single bar chart representing the percentage of facilities

that are being neglected in each district. The bars are color-coded based on

the facility type.

The criterion for a facility to be considered neglected is that it is not visited

by any supervisory officer in the current month as well as the last two

calendar months. Senior officials can easily identify which district has the

highest percentage and take appropriate measures to rectify the situation.

2.2.3. Indicators

The Indicators section in the province level view is very similar to that in the

district level view in terms of layout and structure. The variables are exactly

the same as those in the district level view, as detailed in Tables 1, 2, 3 and 4

in the appendix.

One major difference between the two views, however, is that instead of a

comparison across Tehsils /Towns in a district, the provincial level charts

compare performance across districts for all the indicators.

Also, indicator charts in the province level view contain extra red bars

representing compliance for the BHUs, RHCs and THQs as well as the

DHQs, whereas this is only true for DHQs in the district level view of the

Indicators section. As previously explained, the compliance bars serve as a

gauge of how many visits’ data is used to derive the charts- meaning that the

higher the compliance, the more the value of the variable is reflective of the

actual situation in the corresponding district

Figure 17 depicts a screenshot of the section.

Directorate General Health Services, Health Department, Government of the Punjab

24

Figure 17

This section can prove very useful to track performance of and across

districts in terms of various indicators.

2.2.4. Time Trend Charts

The Time Trend Charts section in the province level view is exactly the

same as that in the district level view, except that there isn’t an extra line for

any district on any of the charts; just a thick line representing the trend of

provincial averages for the same indicators over time, as depicted in Figure

18.

Directorate General Health Services, Health Department, Government of the Punjab

25

Figure 18

As already mentioned, you can move to the district level view if you want a

comparison of the provincial average with a district’s average, or even to the

Tehsil/Town level view if you want a comparison of the district average with

a Tehsil/Town’s average, over time.

Helpline: 0321-4525808

46