epi and vpd surveillance review and post-introduction evaluation

EPI and VPD Surveillance Review and Post-introduction Evaluation

of HPV Vaccine

Report of the Mission Bhutan, 18–30 March 2011

Regional Office for South-East Asia

SEA-Immun-65 Distribution: General

EPI and VPD Surveillance Review and Post-introduction Evaluation

of HPV Vaccine

Report of the Mission Bhutan, 18–30 March 2011


© World Health Organization 2012

All rights reserved.

Requests for publications, or for permission to reproduce or translate WHO publications – whether for sale or for noncommercial distribution – can be obtained from Publishing and Sales, World Health Organization, Regional Office for South-East Asia, Indraprastha Estate, Mahatma Gandhi Marg, New Delhi 110 002, India (fax: +91 11 23370197; e-mail: [email protected]).

The designations employed and the presentation of the material in this publication do not imply the expression of any opinion whatsoever on the part of the World Health Organization concerning the legal status of any country, territory, city or area or of its authorities, or concerning the delimitation of its frontiers or boundaries. Dotted lines on maps represent approximate border lines for which there may not yet be full agreement.

The mention of specific companies or of certain manufacturers’ products does not imply that they are endorsed or recommended by the World Health Organization in preference to others of a similar nature that are not mentioned. Errors and omissions excepted, the names of proprietary products are distinguished by initial capital letters.

All reasonable precautions have been taken by the World Health Organization to verify the information contained in this publication. However, the published material is being distributed without warranty of any kind, either expressed or implied. The responsibility for the interpretation and use of the material lies with the reader. In no event shall the World Health Organization be liable for damages arising from its use.

This publication does not necessarily represent the decisions or policies of the World Health Organization.

Printed in India

iii

Contents Page

Acknowledgements ......................................................................................................................v

Acronyms................................................................................................................................... vii

Executive summary and recommendations................................................................................... ix

1. Background.........................................................................................................................1

Population ..........................................................................................................................2

2. Purpose and methodology of the review .............................................................................2

2.1 Context.......................................................................................................................2

2.2 Objectives of the review .............................................................................................3

2.3 Methodology of the review .........................................................................................4

3. Findings and recommendations...........................................................................................5

3.1 VPD Surveillance ........................................................................................................5

3.2 Immunization system ..................................................................................................8

3.3 Expert Review Committee (ERC) and National Certification Committee (NCC).........12

3.4 Coordination and support .........................................................................................12

Annexes

1. Deployment of review teams and areas visited ..................................................................14

2. Laboratory assessment and review.....................................................................................16

3. HPV Vaccine Post-Introduction Evaluation (PIE) tool results ...............................................19

4. HPV Vaccine Post-Introduction Evaluation (PIE) tool..........................................................93

v

Acknowledgements

Members of the joint national and international review team would like to thank everyone who contributed to the work and outcomes of this review. Our field visits could not have been as productive and informative or as pleasant were it not for the commitment, effort and courteous assistance provided by the Ministry of Health, Royal Government of Bhutan. We would also like to express our appreciation to the WHO Representative and her team for their untiring support, hospitality and overall facilitation.

vii

Acronyms

AEFI adverse event following immunization

AES acute encephalitic syndrome

AFB acid fast bacilli

AFP acute flaccid paralysis

BCG Bacille Calmette-Guérin

BHU Basic health unit

CSF cerbral spinal fluid

DTP Diphtheria-tetanus-pertussis

ELISA enzyme-linked immunosorbent assay

EPI Expanded Programme on Immunization

EQA

ERC

external quality assurance

Expert Review Committee

EVM

GLP

effective vaccine management

good laboratory practice

HIV human immunodefiency virus

HPV human papillomavirus

IEC information, education and communication

ITAG Immunization Technical Advisory Group

JE Japanese encephalitis

JRF Joint Reporting Form

MCH maternal and child health

MCV measles containing vaccine

MLM mid-level manager

MNT maternal neonatal tetanus

MR measles-rubella

NCC National Certification Committee

NPEV non-polio enterovirus

NRHL National Referral Hospital Laboratory

Report of the Mission

viii

OPV oral polio vaccine

PHL public health laboratory

PIE post-introduction evaluation

PPE personal protective equipment

RRHL Regional Referral Hospital Laboratory

SEARO Regional Office for South-East Asia

STI sexually-transmitted infection

TB tuberculosis

TT tetanus toxoid

UNFPA United Nations Population Fund

UNICEF United Nations Children’s Fund

VPD vaccine-preventable disease

VVM vaccine vial monitor

WHO World Health Organziation

ix

Executive summary and recommendations

A team comprising national and international experts reviewed the Expanded Programme on Immunization (EPI), Vaccine Preventable Disease (VPD) surveillance system and Human Papillomavirus (HPV) vaccine introduction in Bhutan from 18-30 March 2011. The general objectives of the review were to determine the status of the programme, quantify achievements in vaccine-preventable disease control and explore ways to improve implementation. With the recent circulation of wild poliovirus in West Bengal, India, the review teams took this as an opportunity to focus on the acute flaccid paralysis (AFP) surveillance indicators and oral polio vaccine coverage. The methodology and tools for the EPI and VPD surveillance portions of the review were adapted from The Common Assessment Tool for Immunization Services1 and the post-introduction evaluation (PIE) tool was provided by WHO headquarters.

In reading the executive summary and recommendations, it is important to remember that there is variation from one district to another. The review team, however attempted to higlight the common challenges observed that were common to all areas visited. One particular challenge was the need for additional supervision and staff training on immunization and surveillance programme activities. This common challenge has led to difficulties in immunization delivery and monitoring as well as surveillance activities. The following are the key findings and recommendations from the review team:

Findings

VPD surveillance:

a surveillance system for vaccine-preventable diseases is in place at all levels and is functioning

norms and standards are defined and available at all levels

VPD surveillance is an integrated system with AFP, measles (rash-illness), neonatal tetanus and acute encephalitic syndrome (AES) existing on the same platform

traditional healers are not formally part of the reporting network

confusion on case definitions/syndromic aproach to VPDs

laboratory and epidemiology case data are not always linked

1 WHO: Immunization, Vaccine and Biologicals (2002). The Common Assessment Tool for

Immunization Services, Geneva, WHO


x

AES sentinal surveillance is not implemented at all sites

inadequate and/or irregular supervision of VPD surveillance by district-level health staff

limited capacity for analysis and interpretation of data especially at the district level – date for action.

Immunization delivery:

systems are in place and able to provide coverage >90% for all antigens

communities are well informed about the importance of childhood immunization and location of services

immunization delivery system is able to absorb new vaccines

an Adverse Event Following Immunization (AEFI) reporting system is in place and has the capacity to handle more cases

updated 2010 immunization guildlines were available, however multiple copies of different previous versions created confusion

lack of standardization of waste management, space is a critical issue in some places

cold chain system intact, however there were concerns about the age of some equipment and future capacity of the network

high-risk migrant groups and nomads were not well enumerated and consequently not adequately targeted for both immunization and surveillance activities

children accompnaying daily wage labourers from India are at a risk for importation of vaccine-preventable diseases (especially wild poliovirus)

concerns about preparedness for the re-introduction of pentavalent in June 2011.

HPV vaccine post-introduction evaluation:

high coverage achieved during 2010 school-based campaigns

piloting conducted in Paro provided important lessons learned for national campaigns

vaccine management, superivsion, transportation and training were well planned and executed during the campaign, which need to be used in the routine immunization programme

IEC materials and messages might be overly complicated for the general public

lack of clarity on policy for girls who missed doses in 2010/drop-outs/mobile populations

lack of clarity on denominator to be used for calculating coverage

EPI and VPD Surveillance Review and Post-introduction Evaluation of HPV Vaccine

xi

concerns about freeze sensitivity and lack of vaccine vial monitor (VVM) on current stock of vaccines

need clarity on sustainability of HPV vaccine after 2015

Recommendations

VPD surveillance:

ensure syndromic approach to VPD surveillance reporting

review and prioritize reporting sites

list reporting sites and review at regular intervals

include traditional healers in the formal reporting network

ensure reporting network captures heatlh seeking behaviour of high-risk groups, migrants and nomads

provide refresher training on VPD/AEFI surveillance, supportive supervision, monitoring and evaluation prioritizing at the district level

scale-up AES surveillance so that it can be used to effectively monitor the re-introduction of pentavalent vaccine in June 2011

improve active case-finding through regular visits (active case search) to reporting units

develop a mechanism for rapidly linking laboratory and epidemiology data

collaborate with Indian districts at border areas to identify migrant, nomadic and high-risk populations.

Immunization delivery:

conduct mid-level management (MLM) training for existing health staff and incumbents focusing on district health officers and assistant district health officers

ensure that adequate training on AEFI management is provided to health staff before and during the re-introduction of pentavalent in June 2011

map high-risk areas and track migrant children

clarify sharps/waste management and universal precaution issues

conduct cold chain risk assessment.

HPV vaccine post-introduction evaluation:

re-assess integration of HPV vaccine into the routine immunization schedule in 12 months


xii

conduct refresher training for EPI staff to transition from campaign to routine immunization

modify/simplify IEC materials and messaging to reach a wider audience

clarify the denominator and calculation of coverage/drop-outs

use freeze tags and cold chain monitoring tools

request next order of HPV vaccine to have VVM

assess sustainability of HPV vaccine after 2015.

1

1. Background

The land locked kingdom of Bhutan, located in the Himalayan mountain range, shares its borders with the Tibetan region of China to the north and the Indian states of Arunachal Pradesh in the east, Assam and West Bengal to the south, and Sikkim to the west. The country is mountainous with more than 72% of the landmass covered by forest. The occasional flat lands are limited to the big river valleys. Bhutan covers an area approximately 38000 square kilometers, roughly 150 kilometres north to south and 300 kilometres east to west. It rises from the plains of India in the south to a height of approximately 7500 metres above sea level in the north. Geographically the area can be divided into three distinct regions – southern sub-tropical, inner Himalayan and the high northern region. The climatic condition in these three regions varies and the rainfall averages from 5000 millimetres per year in the south to 500 millimetres per year in the high northern region. Administratively, the country is divided into 20 Dzongkhags (districts) and 205 gewogs (blocks). Dzongdag is the administrative head of the dzongkhag. Each dzongkhag consists of a number of gewogs, which are formed by a cluster of villages. The headman of the gewog is called a Gup.

Figure 1: Map of Bhutan

Bhutan depends heavily on the road transport network that is linked with the India road system. Towns such as Sipsu, Samtse, Phuentsholing, Gelephu, Sarpang, Pangbang, Daifam, and Samdrup Jongkhar are closest to India and are entry points into Bhutan. Roads from some of these points proceed into the interior of the country, usually following the river valleys and connecting settlements and district centres. See Figure 1.


2

Population

Bhutan is one of the least populated countries in the WHO South-East Asia (SEA) Region. Bhutan reported a population of 683407 with 14605 live births (2009 estimates). A third of the population is below the age of 15 years (211747); and approximately, 85% of the population lives in villages scattered in the arable parts of the country with the remaining living in remote and isolated mountainous areas. Large, northern high altitude areas are virtually empty except for nomadic herders. The urban population is restricted to a few urban centres and townships growing around dzongkhag headquarters. Thimphu in northwest Bhutan is the capital city with a population of approximately 85 000 to 95 000. Other major centres of population are Phuentsholing, Gelephu and Samdrup Jongkhar in the south along the Indian border. These three cities have open borders and are major trading centres of the country. Most Bhutanese still live in extended families with the average family size 5.6 persons.

2. Purpose and methodology of the review

2.1 Context

Justification

Since the last joint national/international review in 2005, which focused on AFP surveillance, Bhutan has completed a measles catch-up campaign and introduced several new vaccines: DTP-Hib-HepB, measles-rubella (MR) and human papillomavirus (HPV). The newest vaccine added to the routine immunization schedule last year was HPV. A vaccine post introduction evaluation (PIE) is usually conducted to assess if there are any programmatic adjustments that are needed. As significant progress towards polio eradication has been made in the SEA Region in the last 24 months, efforts towards ensuring that all Member States are achieving the surveillance standards and documentation is essential. As part of the certification process, and to identify gaps in the integrated surveillance systems, joint national/international reviews are conducted periodically. The basis for polio-free certification is high-quality AFP surveillance. As such, WHO/SEARO has been assisting Member States in strengthening AFP surveillance. As an integral part of this process, countries are encouraged to conduct periodic internal reviews of AFP surveillance, which are complemented by joint national/international surveillance reviews. Bhutan borders the polio- endemic country of India and also shares a large border with China. As there is significant population movement across these borders, there is always the risk of importation of poliovirus from India or other endemic countries. A comprehensive review of the EPI and VPD surveillance programmes has not previously been conducted in Bhutan.


3

2.2 Objectives of the review

The objectives were to:

assess strengths and weaknesses of immunization service delivery at all levels of the health care delivery system

analyse managerial and administrative capacity for immunization at the national and sub-national levels

assess the introduction of new vaccines, particularly HPV

assess strengths and weaknesses of current vaccine distribution mechanisms and cold chain management

assess injection safety and waste management for sharps

review priority setting for immunization programme sustainability

review the capacity of the national surveillance system including laboratory support as applicable to detect and respond to vaccine-preventable diseases (VPD) in a timely manner

document the capacity for surveillance and management of adverse events following immunization (AEFI)

assess training needs for immunization managers, surveillance staff and health workers (vaccinators) at all levels

review the role of the private sector and nongovernmental organizations, as providers of routine immunization services

assess communication strategies, including advocacy, partnership, social mobilization and their implementation

follow-up on the recommendations made at the national/international AFP surveillance review conducted in 2005 and at the Immunization Technical Advisory Group (ITAG) and EPI managers’ meeting

review the activities of the national committees involved in polio eradication (i.e., Expert Review Committee and Laboratory Containment Committee)

The review team aimed to answer the following questions:

Is there capacity for timely and adequate response to VPD outbreaks?

Does the system for the delivery of immunization work?

Does the system for the surveillance of VPD work?

Is AFP, measles, MNT and AES surveillance integrated into VPD surveillance?

Does the laboratory network support VPD surveillance?


4

2.3 Methodology of the review

The review was conducted at national, district and subdistrict levels. Six teams were formed (one central team and five field teams) to review available information and data on the EPI programme at national, district, subdistrict, health center, and other reporting unit levels. Team members interviewed key government officials and individuals involved in the EPI program, VPD surveillance, new vaccine introduction and polio eradication which included: national surveillance officers, district surveillance officers, health-centre staff, beneficiaries, parents, community leaders, laboratory staff, partners and WHO-Bhutan and UNICEF-Bhutan staff. A laboratory review was included as part of the central-level assessment. The HPV PIE was integrated into the EPI/Surveillance review for logistic and administrative efficiency.

Selection of sites

All major cities/municipalities, border areas, and high-risk populations were included in the review. At each level (national, district and subdistrict), the following key technical areas were reviewed: immunization service delivery, injection safety, vaccine-preventable disease surveillance, functioning of the laboratory network, laboratory support for VPDs, vaccine supply and quality including cold chain management, logistics, advocacy, communications and health system support. The review teams made sure that the areas reviewed were representative of all the regions in Bhutan. The review was divided into four components:

(1) orientation (one day) was held in Thimphu where the teams were briefed on the Bhutan health system, EPI/VPD programmes, HPV vaccine introductions and the assessment tools;

(2) field level activity (six days) where each team evaluated two to five districts;

(3) debriefing session (two days) was held in Thimphu where the teams reviewed and consolidated their findings; and

(4) final debriefing (half day) was held in Thimphu with government officials, WHO, UNICEF and UNFPA.

At each health system level the review activities included: interview of surveillance staff, review of hospital records and observation of immunization sessions. Using structured tools developed for EPI, surveillance and HPV vaccine post-introduction, the following technical areas were assessed:

immunization service delivery

injection safety and waste management

vaccine management (supply and cold chain)

vaccine-preventable disease surveillance (focusing on AFP and measles) including AEFI surveillance


5

human resource/training capacity

advocacy and communication.

One team focused specifically on the laboratory component and looked at laboratory capacity and proficiency, global accreditation, logistics and linkages between laboratory and surveillance programmes (see Annex 2 for laboratory assessment).

In addition to the information collected during the review, the following information was also considered: immunization and VPD guidelines, recommendations from the 2005 AFP surveillance review, VPD line listing, aggregated surveillance data, and demographic data. The review team tried to ensure that the areas covered were representative of the different areas of Bhutan.

Limitations of the review

The review team acknowledged the following limitations in the review process:

time and logistics limited the number of districts reviewed

a significant language barrier existed for the international team members, but the national team members contributed significantly in overcoming this barrier.

3. Findings and recommendations

3.1 VPD Surveillance

The vaccine-preventable disease surveillance system began with acute flaccid paralysis surveillance in July 1997 and was expanded to include diphtheria, pertussis, measles, rubella, mumps and tetanus. Tables 1 and 2 show the AFP surveillance indicators and the vaccine-preventable diseases detected in Bhutan, respectively. Bhutan has had variable results over the last couple of years in maintaining the certification standards for AFP surveillance (non-polio AFP rate of 2/100 000 under 15 years of age and adequate stool collection rate of 80%).


6

Table 1 : AFP surveillance indicators, Bhutan, 2001-2009

Indicator 2001 2002 2003 2004 2005 2006 2007 2008 2009AFP cases 4 2 5 3 6 6 4 9 1Wild Polio 0 0 0 0 0 0 0 0 0Compatibles 0 0 0 0 0 0 0 0 0AFP Rate 1.37 0.66 1.66 1 1.99 2.86 1.9 4.28 0.46

Non-Polio AFP Rate2 1.37 0.66 1.66 1 1.99 2.86 1.9 4.28 0.46

Adequate Stool Collection Rate3 50% 0% 33% 100% 33% 33% 50% 67% 0%

Total Stool Samples Tested 5 0 8 6 10 6 2 27 2 % NPEV 0 0 0 0 20 0 0 0 0 % Reported Within 14 Days 100 0 100 100 100 100 100 100 1001 For 2010 data, See the IVD VPD Weekly Bulletin. Data as of 05 July 2009

3 Percent with 2 specimens 24 hours apart and within 14 days of paralysis onset.

2 Number of discarded AFP cases per 100,000 children under 15 years of age.

Table 2: Vaccine-preventable diseases, Bhutan, 2002-2009 (WHO; UNICEF JRF)

Year Diphtheria Pertussis MeaslesTotal

TetanusNeonatalTetanus

Polio Rubella Mumps

2002 0 0 27 0 0 0 ND ND2003 0 0 0 0 0 0 350 ND2004 0 0 3 0 0 0 12 ND2005 0 1 11 1 0 0 81 1442006 0 1 2 59 1 0 11 ND2007 0 0 11 20 0 0 3 ND2008 0 0 7 7 0 0 2 ND2009 0 0 6 23 0 0 15 ND

VPD surveillance

VPD surveillance in Bhutan is an integrated system that includes AFP, measles, MNT and other vaccine-preventable diseases with defined norms and standards. The surveillance network is composed of and includes all government health facilities (hospitals, basic health units and health offices). The review team noted that the health seeking behaviour in Bhutan often initially includes traditional healers. Since traditional healers are not part of the formal surveillance network, vaccine-preventable diseases could be missed or reported late. The surveillance network is currently not prioritized. By prioritizing and expanding the surveillance network, reactivity and sensitivity can be improved. The review team also noted that at district level there was some confusion with case definitions and the sydromic approach to case identification. Again, this approach is important for ensuring a sensitive surveillance network. Systems are in place for reporting laboratory results and linking information with surveillance data; however, the review team noted that delays in linking data made real time decision-making difficult. Ensuring that laboratory data are linked with suspected VPD cases also gives district health officers an opportunity to provide health staff with supportive supervision and feedback.

Even though India has made significant progress towards polio eradication in the last 24 months, maintaining AFP surveillance indicators remains important for all countries.


7

The AFP surveillance indicators are the hallmark for polio-free certification. All countries in the Region need to maintain global standards for the two AFP surveillance indicators. Table 1 shows the AFP surveillance indicators from 2001 to 2009. Nationally, Bhutan has had difficulty in achieving the minimum AFP surveillance indicators for non-polio AFP rate (2/100,000 population <15 years of age) and adequate stool rates (>80%). During the review, the team noted that there were cases in the hospital registers that could be considered AFP and were not reported (i.e. missed cases). Despite the progress made by India, there remains a substantial risk of polio importation. The risk is especially high considering the shift in the location of the most recent wild polio cases in India from the endemic states of Bihar and Uttar Pradesh to West Bengal (a state that borders Bhutan) in addition to the large number of seasonal and migrant India workers in Bhutan. It is therefore extremely important for Bhutan to enumerate these individuals and maintain a consistently sensitive AFP surveillance system across all communities and areas.

Managerial aspects of VPD surveillance

Staffing and training

The level of knowledge of staff at all levels was good. However, there needs to be regular refresher and induction trainings to guarantee a high-quality, functioning surveillance system. The requirements for diseases with elimination and eradication goals are constantly being updated and therefore need to be well understood by health staff.

Monitoring and supervision

The review team observed that the main challenges facing the surveillance programme were related to monitoring and supervision at the district level, which was often irregular and/or inadequate. As new vaccines are added to the immunization schedule, appropriate surveillance will also need to be added, requiring additional monitoring and supportive supervision by district health officers.

Data for action

There is limited capacity to analyse and interpret data at district and basic health centre levels. District health officers should use data collected at the district level to guide case investigations and plan for immunization activities.

Acute encephalitis syndrome (AES) surveillance

Sentinel surveillance for acute encephalitis syndrome (AES) was established in early 2011 as part of a comprehensive plan for the re-introduction of pentavalent immunization in June 2011. It is currently being implemented at five sites. Four sites are sentinel surveillance sites with laboratory facilities: Jigme Dorji Wangchuck National Referral Hospital, Gelephu Regional Referral Hospital, Mongar Regional Referral Hospital and


8

Phuntsholing Hospital. One site, Samdrup Jongkhar Hospital is a sentinel surveillance site without laboratory facilities. AES is defined by fever and altered consciousness (meningitis, cerebral malaria and all forms of encephalitis). District medical officers help collate the line lists and facilitate the transportation of blood and cerebral spinal fluid (CSF) specimens by reverse cold chain. Three of the four sites were visited during the review and the review team noted that AES surveillance had not been fully implemented. There was concern that there may not be enough data to reflect season variation and provide an AES background rate before the re-introduction of pentavalent immunization in June 2011.

Suggestions for improving VPD surveillance

ensure syndromic approach to VPD (particularly important for AFP) surveillance

- consider revising IEC materials and posters for basic health units

review and prioritize reporting sites for VPD surveillance

- list reporting sites and review at regular intervals

- include traditional healers in the surveillance network

- ensure the reporting network captures high-risk, migrant and nomadic populations

provide refresher training at the district level on VPD surveillance, AEFI surveillance, supportive supervision, monitoring and evaluation

scaleup AES surveillance so that it can be used to effectively monitor the re-introduction of pentavalent vaccine in June 2011

improve active case-finding through regular visits (active case search) to reporting sites

develop a mechanism for rapidly linking laboratory and epidemiology data

collaborate with districts in India at border areas for identifying high-risk, migrant and nomadic populations.

3.2 Immunization system

Expanded Programme on Immunization

The Expanded Programme on Immunization (EPI) was launched in the major population centres in 1979 and expanded to all districts in 1980. The EPI activities have been fully integrated into the basic health services and delivered as part of the routine health activities by multi-purpose health workers in the maternal child health (MCH) clinics. Figure 2 shows the progress in routine immunization from 1990 to 2009. The coverage remains high for the six basic childhood immunizations.


9

Figure 2: Estimated routine immunization coverage, Bhutan 1990-2009

Table 3: Percent Immunization Coverage, 1990 - 2009 (WHO/UNICEF Estimated Coverage) Year OPV3 BCG DTP3 MCV1

1990 96 99 96 931991 91 95 91 891992 85 90 86 861993 82 89 79 841994 84 96 86 811995 86 98 87 851996 86 98 87 851997 87 92 87 84

1998 85 94 86 711999 89 90 88 762000 98 97 92 762001 88 81 88 78

2002 89 83 86 782003 96 93 95 882004 90 92 89 872005 95 99 95 93

2006 96 92 95 902007 93 94 95 952008 96 99 96 992009 96 96 96 98

0

20

40

60

80

100

1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009

% C

ove

rag

e

OPV3BCGMCV1DTP3

Source: WHO/UNICEF Joint Reporting Form

The routine immunization schedule has been expanded to include a number of new vaccines. See Table 3 for the current immunization schedule.

Table 3: Routine Immunization Schedule, 2009 (WHO/UNICEFJRF)

Vaccine Age of Administration

BCG Birth

DTP-Hib-HepB 6 weeks, 10 weeks, 14 weeks

OPV Birth, 6 weeks, 10 weeks, 14 weeks

MR 9 months, 24 months

TT 1st pregnancy, +1 month, 2nd pregnancy, 3rd pregnancy, 4th pregnancy, 5th pregnancy

HPV 0 months, +2 months, +6 months

Vitamin A 6 months

The national immunization programme provides a birth dose of BCG; DPT-hepatitis B-Hib combination vaccine with oral polio vaccine at 6, 10 and 14 weeks; and, measles-rubella vaccine at nine and 24 months as well as tetanus toxoid (TT) to pregnant women. Pentavalant vaccine containing DTP, hepatitis B (HepB) and Haemophilus influenzae type B (Hib) antigens was introduced on 1 September 2009 and suspended on 23 October 2009 due to temporally related AEFIs. The re-introduction of pentavalent is scheduled for


10

June 2011. The most recent addition to the routine immunization schedule was HPV vaccine in 2010 and its introduction was specifically evaluation in this review. Table 2 shows the routine immunization coverage since 1990, which has been relatively high and conistent for the last five years.

Immunization service delivery

The health system in Bhutan consists of national referral hospital, regional referral hospitals, district hospitals and basic health units. The review team observed that overall it appears that the immunization system is able to provide coverage greater than 90% for all antigens and able to absorb new vaccines. The review team observed that children in high-risk populations posed a major risk to transporting and/or sustaining vaccine-preventable diseases in Bhutan. These high-risk groups consist of people living and working in areas bordering the Indian states of West Bengal and Assam as well as nomadic populations throughout the country. The children from these high-risk groups were not adequately enumerated and consequently not targeted for both immunization and surveillance activities. Updated immunization guidelines for 2010 were available at many health facilities; however, multiple copies of previous editions created some confusion for health staff.

Suggestions for improving immunization delivery

conduct mid-level manager training (MLM) for existing health staff and incumbents focusing on the district level

map high-risk areas and groups to enumerate and track children for immunization and surveillance activities

ensure that adequate training on AEFI management is provided to health staff before and during the re-introduction of pentavalent in June 2011 (consider including the media in the process)

clarify sharps/waste managment and universal precaution issues.

HPV vaccine: post-introduction evaluation

The HPV vaccine was introduced in 2010 with school based catch-up campaigns with high coverage achieved in all three rounds. Important lessons were learned during the piloting of the vaccine in Paro and allowed for efficient and a well managed roll-out across the country. At the beginning of 2011, HPV vaccine was transitioned to the routine immunization schedule. The review team felt that additional training for health staff at the district level may be required in order to maintain the high coverage achieved during the campaign period. The review team also noted that the IEC material and messaging appeared to be overly complicated and confusing for some girls, parents and basic health care providers. There was a lack of clarity on the denominator to be used for calculating coverage; this was particularly difficult in urban settings. Health-care workers were not


11

clear on the policy for girls that either missed doses in 2010, were drop-outs or part of a mobile population. As HPV is an expensive and temperature-labile vaccine, the review team was concerned that the current lot of vaccines did not have VVMs. A descriptive analysis of the result from the HPV vaccine post-introduction evaluation tool is available in annexes 3 and 4.

Suggestions for HPV vaccine delivery

re-assess integration of HPV vaccine into the routine immunization schedule in 12-18 months

conduct refresher training for EPI staff to ensure smooth transition from campaigns to routine administration

modify/simply IEC materials and messages to reach a wider audience (girls, parents, basic health unit staff)

ensure that next shipment of HPV vaccine has vaccine vial monitors (VVM).

Adverse events following immunization (AEFI)

The AEFI system is functioning: definitions are available, reporting mechanisms are in place, and validation is standardized and documented. The review team observed that the system for reporting serious cases functions with an AEFI committee at the central level that meets to review serious cases for causality assessment. The AEFI system is capable of handling more cases (minor cases) and will be important for additional new vaccine introduction.

Suggestions for improving adverse events following immunizations (AEFI)

conduct refresher training in AEFI management and reporting for staff at the district and health centre levels particularly before the re-introduction of pentavalent in June 2011

encourage robust reporting of AEFIs through the system.

Vaccine supply and cold chain management

The review team did not observe any stock-outs at the district or facility level. The methods for calculating wastage rate and vaccine consumption was not always well understood at the field level. In terms of the cold chain, norms and standards were available for human resources and cold chain material. The review team observed variability in the age of cold chain equipment.


12

Suggestions for improving vaccine supply and cold chain management

conduct cold chain risk assessment and/or effective vaccine management (EVM) evaluation

clarify the source of population data for the denominator that is used to calculate coverage and drop-out rates

use freeze tags and/or other cold chain monitoring tools to ensure vaccine efficacy.

3.3 Expert Review Committee (ERC) and National Certification Committee (NCC)

As part of the review of the central level, the review team looked at the composition of the Expert Review Committee (ERC) and the National Certification Committee (NCC). The technical composition of the committee was appropriate. The NCC is providing SEARO with annual updates on certification and is actively participating in regional meetings as required.

Suggestion for improving the ERC and NCC

review membership of these two bodies regularly (annually) to ensure continued transparency and independence.

3.4 Coordination and support

WHO-SEARO

The recommendations for WHO-SEARO focus on providing support to the WHO-Bhutan country office to assist in implementation of the recommendations outlined in this report. The review team felt that the regional and country teams had a lot to contribute to assisting the government of Bhutan to maximize supervision and training opportunities. The Government of Bhutan has made a substantial investment in maintaining and expanding the surveillance network from just AFP surveillance to include measles, neonatal tetanus and acute encephalitic syndrome (AES).

Suggestions for improving coordination and support from WHO-SEARO and WHO-Bhutan

support mid-level manager (MLM) training for existing health staff and incumbents

support refresher training for EPI staff at the district level focusing VPD surveillance and supportive supervision


13

facilitate regular cross-border meetings between the government of Bhutan and the government of India (at the district level) to review progress and coordinate VPD surveillance and immunization activities

conduct a formal follow-up in 18-24 month to review the implementation of these recommendations.


14

Annex 1

Deployment of review teams and areas visited

Table 5: Joint review team

Team International Team Member National Team Member(s) Districts

1 Dr Sunil Bahl Dr Dhrupthob Sonam (TL)

Dr Sonam Ugen

Haa

Chukha

Samtse

2 Dr Prashanta Roy Mr Sonam Zangpo (TL)

Dr Chandralal Mogar

Tsirang

Sarpang

Zhemgang

Trongsa

Wangduphodrang

3 Dr Abhijeet Anand Mr Wangchuk Dukpa (TL)

Dr Sangay Phuntsho

Trashigang

Mongar

4 Dr Ganga Choudhary Dr Sonam Tshering (TL) Pema Gatshal

S/Jongkhar

5 Dr Madhav Ram Dr KP Tshering (TL)

Mr Sonam Dorji

Trashi Yangtse

Bumthang

Punakha

Wangduphodrang

S Dr Susan Wang Mr Dorji Phub

Mr Tshewang Dorji Tamang

HPV PIE

S Dr Patrick O’Connor Mr Dorji Phub Supervisory/briefings


15

Figure 5: Districts visited by the joint review team

1

1 1

23

4 4

3

5

2

2 22

555

S


16

Annex 2

Laboratory assessment and review

The laboratory network was assessed as part of the EPI and VPD surveillance reivew. Dr Nalini Ramamurthy from IVD-SEARO conducted the laboratory assessment with Mr. Sonam Wangchuk and Mr Dorji Phuntsho from 14-18 March 2011. The team visited the following facilities (see Figure 6 for map of the districts visited):

Thimphu

- health information management system unit

- public health laboratory

- department of laboratory services (JDWNRH)

Gelephu, Sarpang Dzongkhag

- central laboratory Gelephu

- district hospital Sarpang

- basic health unit (BHU) Jigmeling

Phuntsholing, Chukha Dzongkhag

- hospital laboratory.

The laboratory review consisted of the following activities: review of VPD data at the national and sub-national level, discussions with medical and laboratory focus points at the facilities visited and a physcial assessment of the laboratories to include infrastructure, manpower, diagnostic services, waste management, data management and quality assurance system.

Laboratory capacity

The review team found that the laboratory network was functioning well from the primary to tertiary heatlh care level with adequately trained personnel for the existing work load. The facilities are commensurate with the level of testing expected at each level:

basic health unit (BHU) laboratory: malaria microscopy, routine urine and blood grouping

district laboratory: all BHU laboratory procedures plus AFB microscopy, gram stain, stool examination, serology for rapid test kits


17

NRHL/RRHL: all district laboratory procedures plus histopathology, cytology, biochemistry and bacterial culture

PHL: national WHO accredited laboratory for measles, rubella, Japanese encephalitis (JE), tuberculosis (TB), HIV and sexually transmitted infections (STI).

For serology, ELISA-based tests (which are more economical) should be considered compared to rapid test kits. ELISA can be easily done at the RRHL with additional equipment and training. For bacteriology, there is adequate capacity for diagnosis of VPD (Hib, streptococcus, pneumococcus, meninigitis) at the NRHL, RRHL and PHL. The PHL has the capacity for molecular testing with additional training. A dedicated bacterial bio-safety cabinet is essential for a clean work space inorder to prepare media/reagents and avoid cross contaimination.

The review team observed delays in biochemical test reporting due to limited equipment at the GRRH and Phuntsholing hospital laboratory. An additional photometer or autoanalyser could help with this problem. Biochemical testing is currently not being performed at the district level but should be considered during future expansion of laboratory services.

Laboratory space

The review team found that there was adequate space at the national level; however there was concern at the sub-national levels (regional referral laboratory and district hospital laboratory). There was inadequate space, overcrowding/inappropriate placement of equipment that compromises best laboratory practices. The PHL in Thimphu at present has adequate space but additional space will be needed if more functions will be added to their responsibilities (quality assurance functions, research and molecular testing). Except at GDWNRH in Thimphu, the segregation and disposal of laboratory waste was not strictly followed as per national guidelines. All laboratories are using safety boxes for the disposal of sharps and needles; however, boxes (containing biological wastes) are not decontamined in laboratories before they are put in disposal pits.

Laboratory procurement/equipment/procedures

The review team observed that reagents and test kits were in short supply at all levels leading to an interruption of testing abilities. Equipment log books were available and systematically being maintained. The scheduling of preventive maintenance for critical equipment should be stream-lined to ensure unterruped testing capacity. Standard operating procedures (SOPs) were not available and need to be prepared and strictly followed as per good laboratory practice (GLP). External quality assurance (EQA) programme is well established and coordinated by the PHL for HIV, TB and malaria microscopy. EQA could be extended to include other bacterial and serologic tests, which will allow for standardization and comparability of results. Quality control and assurance procedures should be reinforced and strengthened through regular laboratory management training.


18

Figure 6: Districts visited by laboratory reivew team

Suggestions for improving coordination and support from the laboratory network

conduct a comprehensive review of waste disposal mechansims at all levels including hospital waste management

- provide autoclave for decontamination of waste

- conduct training on personal protective equipment (PPE), biosafety and waste disposal

review the responsibilities and workload of the PHL to strengthen and expand diagnositc services for other vaccine-preventable diseases and biochemical testing

ensure adequate procurement and maintenance of laboratory equipment

- streamline procurement of diagnostic kits particularly those with a short shelf-life to avoid stock outs

- schedule periodic preventive maintenance for critical equipment

develop quality assessment and monitoring tools for laboratories at the national level

develop training plan for laboratory technologist and technicians on good laboratory practices, quality assurance, biosafety and data management/analysis.


19

Annex 3

HPV Vaccine Post-Introduction Evaluation (PIE) tool results

Frenquency table for central level: 1 result (please refer to annex 4 for the data dictionary and the exact questionnaire)

Question/Field Response

Number of responses 1

Level Central Level

C1 5-May-10

C2_VacIntro National Introduction

C3_GPop12y

C4_IntroDecision Strong political will; ACCF+donation; benefit+cost; sustainability

C5_NIACSupport Yes

C6_CurImmSch EPI Manual

C7_ImmSchChanged Yes, previously only vaccinated upto 2 yr old.

C8_HPVSchedule Dose1=0; Dose2=2; Dose3=6

C9_HPVLinkedAdoles No

C10_NatlStrategy No

C11_CervScreenChanged No

C12_DiseasePrevent Cervical Cancer + genital wart

C13_VacIntroPlan Yes, both national and district plan/timeline

C14_Support Yes, Technical, Financial supports

C15_TrainingAudience DMOs+DHOs from all districts at national level

C15_TrainingType Cascade

C15_TrainingBefore Yes

C15_TrainingAfter

C15_TrainingDays 2 Days

C15_ConductedBy DHOs+DMOs

C16_Financed Much

C17_TrainingImprove Yes, better to have 2 batches of trainings for larger number of people; educational guide for BHUs

C18_MaterialProvided HPV introduction guide only


20


C19_Consent Nat'l 2010, not in 2011

C20_issue Yes, people were concerned

C21_MethodsUsed School health coordinator

C22_ImmDatabase Not yet, will be separate for several years since printed many form

C23_Formula No. doses administered/No. target population * 100

C24_Rubella

C25_Coverage

C26_FormsSubmited BHU report monthly to district; district report quarterly to central

C27_ChangeColdChain 2010 - new cold room 600 of Bhutan purchased. This was because 1) HPV vaccine, 2) to separate reagents

C28_Problem No Problems

C29_FreezeWatch

C30_Policy Yes, modifying

C31_VaccineForcast

C32_Estimated

C33_VacOrder

C34_VacTransport Mongar+Gelephu EPI van deliver vaccines to hospital

C35_VacSendOut Quarterly, vaccine + needles delivered

C36_FreqChange Yes, for campaign 2-3 more shipments. Now goes with other EPI vaccines

C37_Outreach More vaccine carriers

C38_Effect No

C39_Cost For campaign extra transport but not this year

C41_StockOut Yes, September 2010 for 1 week

C42_VacExpire Yes, TT, OPV

C43_VVM

C44_InjSupp It depends

C45_Waste By infection control

C46_FollowGuideline Yes

C47_ChangeWaste Don't think so

C48_ChangeGuideline No

C49_WasteCalculate

C50_HPVWaste

C51_ReduceWaste No


21


C52_FreqVisit Twice per year not all district. Twice per year not all BHUs. Depends on districts

C53_SupVisit Yes, Chunkha in Oct 2010 and Mongar in Sep 2010

C54_Feedback Written supervisory checklist; Oral discussion with staff

C55_Issues Target population not updated at district level; No particular MCH incharge of clinic - identify MCH incharge at district hospital; Should document expiry date of AD syringes

C56_FollowupVisit Depends on time

C57_ReceivedSupVisit No

C58_AEFIMonitoring Yes, incorporated into EPI manual

C59_CrisisPlan Yes

C60_ChangeProtocal Same

C61_AEFIReported Yes, not serious but for other vaccines

C62_Lunch Yes, Thimpu at School with Queen Mother, ACCF

C63_MediaOutlet Radio, TV, Community groups, Celebrity, Government officials

C64_Materials Posters, Brochures, banners

C65_BudgetVaccine No, for routine or for HPV only, co-financing for tetra/penta

C66_EPIVacFinanced 2008 JCV started funding all traditional vaccine, before it was by JICA

C67_HPVVacPaid 2010-2011 = Merch; 2012-2015=ACCF

C68_Operational 2010; Merch-$150,000, ACCF-$150,000, UNFPA-$50,000

C69_NewVaccIntro Birth dose HepB in May-June 2011; Penta in June 2011; Rota and PCV after study

C70_VacAssessment Don't know, depends on IARC work+cancer registry department

C71_HPVAccepted Yes; by health-care workers, professional societies, government, media

C72_ColdChain Yes, district hospital needed vaccine from regional stores more often for campaign

C72_Trasnport Yes

C72_Wastage No

C72_Comm Yes

C72_Training Yes

C72_other Yes, meetings, time

C73_Effect Improved the EPI programme

C74_HPVIntroduction Smooth, minor problems.


22


C75_Advice Plan well in advance, also for budget, have good guidelines, reference; have advocacy material+adequate training; Depends on country site on whether a pilot project useful, can give more confidence.

C76_freezer Yes

C77_ThermoOutside Yes

C78_ThermoInside Yes

C79_temperature Yes

C80_Log No

C81_TempRecord Twice daily

C82_Weekends Yes

C83_FEFO Yes

C84_ExipredVac No

C85_VVMUsable yes, all vaccines usable

C86_VVMStage2 Yes

C87_Space Yes

C88_InjEquip Yes, Adequate Space, clean and dry condition and well organized.

Frenquency tables for district level (please refer to annex 4 for the data dictionary and the exact questionnaire)

District

Frequency Percentage Valid

percentage Cumulative percentage

2 25.0 25.0 25.0

PUNAKHAA 1 12.5 12.5 37.5

SARPANG 1 12.5 12.5 50.0

Trangsa Hospital 1 12.5 12.5 62.5

TSIRANG 1 12.5 12.5 75.0

WANGDUE 1 12.5 12.5 87.5

ZHEMGANG 1 12.5 12.5 100.0

Valid

Total 8 100.0 100.0


23

D1_DateVac



04-May-2010 1 12.5 16.7 16.7

05-May-2010 4 50.0 66.7 83.3

22-May-2010 1 12.5 16.7 100.0

Valid

Total 6 75.0 100.0

Missing System 2 25.0

Total 8 100.0

D2_GPop12-18y



1 12.5 12.5 12.5

218 1 12.5 12.5 25.0

2206 1 12.5 12.5 37.5

3037 1 12.5 12.5 50.0

5025 1 12.5 12.5 62.5

931 1 12.5 12.5 75.0

NA 2 25.0 25.0 100.0

Valid

Total 8 100.0 100.0

D3_Prevent



1 12.5 12.5 12.5

CaCx 6 75.0 75.0 87.5

Cancer 1 12.5 12.5 100.0

Valid

Total 8 100.0 100.0


24

D4_Plan-Timeline

Frequency Percentage Valid percent Cumulative percentage

Yes, both National and District plan/timeline

1 12.5 12.5 12.5

Yes, District plan/timeline 4 50.0 50.0 62.5

Yes, District plan/timeline, No IEC plan

1 12.5 12.5 75.0

Yes, National plan/timeline 2 25.0 25.0 100.0

Valid

Total 8 100.0 100.0

D5_TrainingAudience

Frequency percent Valid

percentage Cumulative

percent

Doctors, Nurses, HCW 2 25.0 25.0 25.0

Doctors, Nurses, HCW, BHWs

1 12.5 12.5 37.5

Nurses, HCW 2 25.0 25.0 62.5

Nurses, HCW, BHU staff 1 12.5 12.5 75.0

Nurses, HCW, Pharmacy Technician

1 12.5 12.5 87.5

Nurses, HCW, Pharmacy Technician, Lab technician

1 12.5 12.5 100.0

Valid

Total 8 100.0 100.0


25

D5_TrainingType



Cascade 3 37.5 37.5 37.5

Cascade, Region-by Region 1 12.5 12.5 50.0

District Level 2 25.0 25.0 75.0

One time training 2 25.0 25.0 100.0

Valid

Total 8 100.0 100.0

D5_TrainingBeforeIntro

Frequency Percentage Valid percentage Cumulative percentage

Valid Yes 8 100.0 100.0 100.0

D5_ifYesHowLongBefore

Frequency Percentage Valid percent Cumulative percentage

1 12.5 12.5 12.5

1 day 1 12.5 12.5 25.0

2 weeks 1 12.5 12.5 37.5

3 days 1 12.5 12.5 50.0

3 weeks 1 12.5 12.5 62.5

4 weeks 2 25.0 25.0 87.5

6 weeks 1 12.5 12.5 100.0

Valid

Total 8 100.0 100.0


26

D5_TrainingAfterIntro


2 25.0 25.0 25.0

No 5 62.5 62.5 87.5

Yes 1 12.5 12.5 100.0

Valid

Total 8 100.0 100.0

D5_ifYesHowLongAfter



7 87.5 87.5 87.5

6 weeks after 1st round 1 12.5 12.5 100.0

Valid

Total 8 100.0 100.0

D5_TrainingDay


2 25.0 25.0 25.0

1 day 3 37.5 37.5 62.5

2 days 3 37.5 37.5 100.0

Valid

Total 8 100.0 100.0

D5_WhoConducted


1 12.5 12.5 12.5

MO 3 37.5 37.5 50.0

MO & DHO 4 50.0 50.0 100.0

Valid

Total 8 100.0 100.0


27

D6_TrainingFinanced



1 12.5 12.5 12.5

Adequate budget provided by MOH

1 12.5 12.5 25.0

Central 2 25.0 25.0 50.0

MOH 2 25.0 25.0 75.0

MOH sent the fund 1 12.5 12.5 87.5

RGoB 1 12.5 12.5 100.0

Valid

Total 8 100.0 100.0

D7_TrainingImproved



Don't Know 1 12.5 12.5 12.5

No 5 62.5 62.5 75.0

Yes, all health workers 1 12.5 12.5 87.5

Yes, refresher training 1 12.5 12.5 100.0

Valid

Total 8 100.0 100.0

D8_EduMaterial



EPI manual for health worker 1 12.5 12.5 12.5

Guidelines 1 12.5 12.5 25.0

HIV (vaccination) manual for health worker

1 12.5 12.5 37.5

HPV Manual 3 37.5 37.5 75.0

Training Manual 1 12.5 12.5 87.5

Training module, HPV manual, PowerPoint

1 12.5 12.5 100.0

Valid

Total 8 100.0 100.0


28

D9_ConsentRequired



No 4 50.0 50.0 50.0

Yes 4 50.0 50.0 100.0

Valid

Total 8 100.0 100.0

D10_IfYesPoseAnyIssue



3 37.5 37.5 37.5

No 3 37.5 37.5 75.0

Yes, people worried 1 12.5 12.5 87.5

Yes, some girls were reluctant when consent was required. But did not hesitate when it was not required.

1 12.5 12.5 100.0

Valid

Total 8 100.0 100.0

D11_MethodsUsed



2 25.0 25.0 25.0

Advocacy, IEC, IPC card 1 12.5 12.5 37.5

Card 1 12.5 12.5 50.0

Media, TV, posters, brochures, ORC session, public meeting

1 12.5 12.5 62.5

Register 1 12.5 12.5 75.0

Telephone 1 12.5 12.5 87.5

VHW, HWs during 1st and 2nd round

1 12.5 12.5 100.0

Valid

Total 8 100.0 100.0


29

D12_DatabaseUpdated



Don't Know 2 25.0 25.0 25.0

No 4 50.0 50.0 75.0

Yes 2 25.0 25.0 100.0

Valid

Total 8 100.0 100.0

D13_Formula



2 25.0 25.0 25.0

No of beneficiaries * 0.25 / No of beneficiaries

1 12.5 12.5 37.5

Standard 1 12.5 12.5 50.0

Total immunized / Target population * 100

4 50.0 50.0 100.0

Valid

Total 8 100.0 100.0

D14_RubellaCoverage



7 87.5 87.5 87.5

Not available, 2006 1 12.5 12.5 100.0

Valid

Total 8 100.0 100.0


30

D15_FirstDose


78.4 1 12.5 16.7 16.7

89.0 1 12.5 16.7 33.3

97.0 1 12.5 16.7 50.0

98.8 1 12.5 16.7 66.7

99.0 1 12.5 16.7 83.3

100.0 1 12.5 16.7 100.0

Valid

Total 6 75.0 100.0


Total 8 100.0

D15_SecondDose


90 1 12.5 16.7 16.7

96 1 12.5 16.7 33.3

97 1 12.5 16.7 50.0

99 2 25.0 33.3 83.3

100 1 12.5 16.7 100.0

Valid

Total 6 75.0 100.0


Total 8 100.0


31

D15_LastDose


81 1 12.5 14.3 14.3

96 1 12.5 14.3 28.6

97 2 25.0 28.6 57.1

98 2 25.0 28.6 85.7

99 1 12.5 14.3 100.0

Valid

Total 7 87.5 100.0


Total 8 100.0

D16_ChangeColdChain



1 12.5 12.5 12.5

44 thermometer, 3 vaccine carriers additional

1 12.5 12.5 25.0

No 6 75.0 75.0 100.0

Valid

Total 8 100.0 100.0

D17_PolicyVacMgt


Valid Yes 8 100.0 100.0 100.0


32

D17_IfYesUpdated



3 37.5 37.5 37.5

2007 guidelines, but it does not include HPV

1 12.5 12.5 50.0

But not yet distributed 1 12.5 12.5 62.5

No 1 12.5 12.5 75.0

Separate guideline for HPV 1 12.5 12.5 87.5

Yes 1 12.5 12.5 100.0

Valid

Total 8 100.0 100.0

D18_EstimateChange


No 5 62.5 62.5 62.5

Yes 3 37.5 37.5 100.0

Valid

Total 8 100.0 100.0

D19_WasteDisposal



As per the manual 1 12.5 12.5 12.5

Burn and bury at all levels 1 12.5 12.5 25.0

Injection after use are put into safety box and burn

1 12.5 12.5 37.5

Syringes/needles in safety box, pit burn 2 25.0 25.0 62.5

Waste disposal pit, burn & burry 1 12.5 12.5 75.0

Yes 2 25.0 25.0 100.0

Valid

Total 8 100.0 100.0


33

D20_FollowGuideline


No 1 12.5 12.5 12.5

Yes 7 87.5 87.5 100.0

Valid

Total 8 100.0 100.0

D21_ChangeWasteDisp


1 12.5 12.5 12.5

No 7 87.5 87.5 100.0

Valid

Total 8 100.0 100.0

D22_ChangeInjectSafety


No 7 87.5 87.5 87.5

Yes 1 12.5 12.5 100.0

Valid

Total 8 100.0 100.0

D23_formulaVaccWaste



3 37.5 37.5 37.5

(Total doses - doses used)/Total Doses * 100

1 12.5 12.5 50.0

EPI formula used 1 12.5 12.5 62.5

No of vaccine given / No of vaccine doses issued * 100

1 12.5 12.5 75.0

Same formula 1 12.5 12.5 87.5

Vaccine wastage not calculated 1 12.5 12.5 100.0

Valid

Total 8 100.0 100.0


34

D24_WastageRate


4 50.0 50.0 50.0

?0 1 12.5 12.5 62.5

0 3 37.5 37.5 100.0

Valid

Total 8 100.0 100.0

D25_ReduceWR


1 12.5 12.5 12.5

No 7 87.5 87.5 100.0

Valid

Total 8 100.0 100.0

D26_Regional


0 1 12.5 33.3 33.3

1 1 12.5 33.3 66.7

2 1 12.5 33.3 100.0

Valid

Total 3 37.5 100.0


Total 8 100.0


35

D26_District



6 75.0 75.0 75.0

5-6 times 1 12.5 12.5 87.5

Once in each round 1 12.5 12.5 100.0

Valid

Total 8 100.0 100.0

D26_HealthFacility



4 50.0 50.0 50.0

2 1 12.5 12.5 62.5

Once in each round 1 12.5 12.5 75.0

Quarterly 1 12.5 12.5 87.5

Twice a year 1 12.5 12.5 100.0

Valid

Total 8 100.0 100.0

D27_SupVisit


Valid Yes 8 100.0 100.0 100.0


36

D27_IfYesHowOften



By medical officer 1 12.5 12.5 12.5

by Medical superintendent 1 12.5 12.5 25.0

During all 3 rounds 1 12.5 12.5 37.5

During the campaign by MO/DHO 1 12.5 12.5 50.0

Once, by DHO 1 12.5 12.5 62.5

Some BHU quarterly, some not done

1 12.5 12.5 75.0

Three months 1 12.5 12.5 87.5

Yes, every round 1 12.5 12.5 100.0

Valid

Total 8 100.0 100.0

D28_Feeback



Discussion with staff 1 12.5 12.5 12.5

Send site visit report 1 12.5 12.5 25.0

Send site visit report, discussion with staff

2 25.0 25.0 50.0

Supervisory Checklist, discussion with staff

1 12.5 12.5 62.5

Supervisory Checklist, Send site visit report, discussion with staff

1 12.5 12.5 75.0

Supervisory logbook, oral 1 12.5 12.5 87.5

Visitor's register, discussion with staff, telephone

1 12.5 12.5 100.0

Valid

Total 8 100.0 100.0


37

D29_IssueHPV



1 12.5 12.5 12.5

At some sites, there was space issue, small room

1 12.5 12.5 25.0

Disposal, documentation 1 12.5 12.5 37.5

Many girls were reluctant to receive second dose due to AEFI in others

1 12.5 12.5 50.0

No 3 37.5 37.5 87.5

No problem 1 12.5 12.5 100.0

Valid

Total 8 100.0 100.0

D30_FupVisit



No 4 50.0 50.0 50.0

No (no inadequate performance)

1 12.5 12.5 62.5

Yes 3 37.5 37.5 100.0

Valid

Total 8 100.0 100.0

D31_SupervisoryVisit



No 3 37.5 37.5 37.5

Yes 3 37.5 37.5 75.0

Yes, national supervisor for HPV campaign

2 25.0 25.0 100.0

Valid

Total 8 100.0 100.0


38

D32_MonitoringAEFI


No 2 25.0 25.0 25.0

Yes 6 75.0 75.0 100.0

Valid

Total 8 100.0 100.0

D33_CrisisPlan



Guideline 1 12.5 12.5 12.5

NA 1 12.5 12.5 25.0

No 3 37.5 37.5 62.5

Yes, all vaccinators carry emergency kits

1 12.5 12.5 75.0

Yes, emergency kit 2 25.0 25.0 100.0

Valid

Total 8 100.0 100.0

D34_ChangeProtocal


1 12.5 12.5 12.5

NA 1 12.5 12.5 25.0

No 6 75.0 75.0 100.0

Valid

Total 8 100.0 100.0


39

D35_AEFIReported



Don't know 1 12.5 12.5 12.5

No 6 75.0 75.0 87.5

Yes, 14 (HPV) had giddiness, fainting and they were treated in the hospital

1 12.5 12.5 100.0

Valid

Total 8 100.0 100.0

D36_OfficialLunch



No 4 50.0 50.0 50.0

Yes 4 50.0 50.0 100.0

Valid

Total 8 100.0 100.0

D37_Media



Community groups, Other - school health /principals

1 12.5 12.5 12.5

DYT, school 1 12.5 12.5 25.0

Radio, TV 1 12.5 12.5 37.5

Radio, TV, community groups, government officials

1 12.5 12.5 50.0

Radio, TV, community groups, HWs, government officials

1 12.5 12.5 62.5

Radio, TV, government officials, HWs

1 12.5 12.5 75.0

TV 1 12.5 12.5 87.5

TV, community groups, local Media, newspaper, district level meeting

1 12.5 12.5 100.0

Valid

Total 8 100.0 100.0


40

D38_HEMaterial



No 1 12.5 12.5 12.5

Posters 1 12.5 12.5 25.0

Posters, Brochures 5 62.5 62.5 87.5

Posters, Brochures, Flyers 1 12.5 12.5 100.0

Valid

Total 8 100.0 100.0

D39_HCW



Valid Yes 8 100.0 100.0 100.0

D39_Societies



1 12.5 12.5 12.5

Yes 7 87.5 87.5 100.0

Valid

Total 8 100.0 100.0

D39_Community


1 12.5 12.5 12.5

Yes 7 87.5 87.5 100.0

Valid

Total 8 100.0 100.0

D39_Govt


Valid Yes 8 100.0 100.0 100.0


41

D39_Media



1 12.5 12.5 12.5

Yes 7 87.5 87.5 100.0

Valid

Total 8 100.0 100.0

D40_ColdChain



2 25.0 25.0 25.0

No 6 75.0 75.0 100.0

Valid

Total 8 100.0 100.0

D40_VacTrans



2 25.0 25.0 25.0

No 6 75.0 75.0 100.0

Valid

Total 8 100.0 100.0

D40_Wastage



2 25.0 25.0 25.0

No 6 75.0 75.0 100.0

Valid

Total 8 100.0 100.0


42

D40_ComMaterial



3 37.5 37.5 37.5

No 5 62.5 62.5 100.0

Valid

Total 8 100.0 100.0

D40_Training



2 25.0 25.0 25.0

No 6 75.0 75.0 100.0

Valid

Total 8 100.0 100.0

D40_Other



2 25.0 25.0 25.0

No 6 75.0 75.0 100.0

Valid

Total 8 100.0 100.0

D41_Effect



Valid No effect 8 100.0 100.0 100.0


43

D42_Process



Smooth - minor problem 1 12.5 12.5 12.5

Smooth - minor problem (IEC materials)

1 12.5 12.5 25.0

Very smooth - no problem 6 75.0 75.0 100.0

Valid

Total 8 100.0 100.0

D43_Advice



2 25.0 25.0 25.0

Awareness creation about the benefit of vaccine, even parents to be well informed, high level political commitment, inter-sectoral collaboration.

1 12.5 12.5 37.5

Awareness must be thoroughly discussed

1 12.5 12.5 50.0

Coordination from school, avoid written consent

1 12.5 12.5 62.5

Inform public about its benefits 1 12.5 12.5 75.0

Mass awareness essential, community leaders should be involved, commitment of HW

1 12.5 12.5 87.5

Proper awareness among the public.

1 12.5 12.5 100.0

Valid

Total 8 100.0 100.0


44

Frenquency tables for health facility level (please refer to annex 4 for the data dictionary and the exact questionnaire)

District/HF



Bajo BHU - 1 1 5.3 5.3 5.3

Changangkha School 1 5.3 5.3 10.5

Chapaha BHU 1 5.3 5.3 15.8

Damphu Hospital, CHU 1 5.3 5.3 21.1

Gyelposing BHU 1 5.3 5.3 26.3

Jigmecholing BHU 1 5.3 5.3 31.6

JWDRNH 1 5.3 5.3 36.8

Kabesa BHU 1 5.3 5.3 42.1

Mendvelgap BHU - II 1 5.3 5.3 47.4

Nobgang BHU 1 5.3 5.3 52.6

Phangny BHU 1 5.3 5.3 57.9

Punakha 1 5.3 5.3 63.2

Radi BHU 1 5.3 5.3 68.4

SARPANG 1 5.3 5.3 73.7

Tongtophey BHU 1 5.3 5.3 78.9


Trongsa 1 5.3 5.3 89.5

Wangdue 1 5.3 5.3 94.7

Yebitaplsa Hospital 1 5.3 5.3 100.0

Valid

Total 19 100.0 100.0


45

TypeHF



3 15.8 15.8 15.8

BHU 2 10.5 10.5 26.3

Gelephu Hospital, CHU 1 5.3 5.3 31.6

Health Centre/Clinic 11 57.9 57.9 89.5

Hospital Clinic 1 5.3 5.3 94.7

School 1 5.3 5.3 100.0

Valid

Total 19 100.0 100.0

H1_Working



1 5.3 5.3 5.3

No 1 5.3 5.3 10.5

Yes 17 89.5 89.5 100.0

Valid

Total 19 100.0 100.0

H2_DateVacIntro


percentage Valid

percentage

05-May-2010 14 73.7 87.5 87.5

13-Mar-2011 1 5.3 6.3 93.8

05-May-2011 1 5.3 6.3 100.0

Valid

Total 16 84.2 100.0


Total 19 100.0


46

H3_NumTrained


percentage Valid

percentage

1 2 10.5 11.8 11.8

2 7 36.8 41.2 52.9

3 4 21.1 23.5 76.5

6 2 10.5 11.8 88.2

12 1 5.3 5.9 94.1

16 1 5.3 5.9 100.0

Valid

Total 17 89.5 100.0


Total 19 100.0

H3_WhoTrained


percentage Valid

percentage

5 26.3 26.3 26.3

ACO, Nurse 1 5.3 5.3 31.6

All HWs 1 5.3 5.3 36.8

BHW, HA, Nurses 1 5.3 5.3 42.1

BHW, HA, Pharmacy Technician

1 5.3 5.3 47.4

HA 2 10.5 10.5 57.9

HA, BHW 6 31.6 31.6 89.5

Medical Officer 1 5.3 5.3 94.7

Vaccinators 1 5.3 5.3 100.0

Valid

Total 19 100.0 100.0


47

H3_NumWorking


percentage Valid

percentage

1 2 10.5 11.8 11.8

2 7 36.8 41.2 52.9

3 3 15.8 17.6 70.6

4 1 5.3 5.9 76.5

6 2 10.5 11.8 88.2

9 1 5.3 5.9 94.1

16 1 5.3 5.9 100.0

Valid

Total 17 89.5 100.0


Total 19 100.0

H3_TrainingDays


percentage Valid

percentage

1 day 12 63.2 63.2 63.2

2 days 5 26.3 26.3 89.5

3 days 2 10.5 10.5 100.0

Valid

Total 19 100.0 100.0


48

H3_Topics


percentage Valid

percentage

1 5.3 5.3 5.3

As per manual 4 21.1 21.1 26.3

Cold chain, vaccine safety, vaccine wastage, coverage

1 5.3 5.3 31.6

Disease it prevents, schedule, AEFI reporting

1 5.3 5.3 36.8

Disease it prevents, schedule, AEFI, emergency kit, reporting

1 5.3 5.3 42.1

Disease, HPV, AEFI monitoring 1 5.3 5.3 47.4

Genital warts, cervical cancer 1 5.3 5.3 52.6

HPV New vaccine, AEFI 1 5.3 5.3 57.9

HPV vaccination, target age, side effects, manage, keep for 15-30 mins

1 5.3 5.3 63.2

HPV vaccine, AEFI, immunization schedule, safety, disease prevented

2 10.5 10.5 73.7

HPV vaccine, AEFI, immunization schedule, safety, waste management

1 5.3 5.3 78.9

New vaccine introduction 1 5.3 5.3 84.2

No. of doses, AEFI, disease prevented, reporting

1 5.3 5.3 89.5

Target girls, AEFI, disease it prevents, Reporting

1 5.3 5.3 94.7

Target population, transmission of HPV, prevention, recording and reporting

1 5.3 5.3 100.0

Valid

Total 19 100.0 100.0


49

H3_Skill



13 68.4 68.4 68.4

90degree, IM injection 1 5.3 5.3 73.7

Yes 5 26.3 26.3 100.0

Valid

Total 19 100.0 100.0

H3_TrainedOther



3 15.8 15.8 15.8

No 13 68.4 68.4 84.2

Yes 3 15.8 15.8 100.0

Valid

Total 19 100.0 100.0

H3_TrainingBeforeVaccine



No 1 5.3 5.3 5.3

Yes 18 94.7 94.7 100.0

Valid

Total 19 100.0 100.0


50

H3_IfYesHowLongBefore



5 26.3 26.3 26.3

1 day 1 5.3 5.3 31.6

1 month 2 10.5 10.5 42.1

2 weeks 5 26.3 26.3 68.4

3 weeks 1 5.3 5.3 73.7

4 weeks 3 15.8 15.8 89.5

6 weeks 1 5.3 5.3 94.7

Few months 1 5.3 5.3 100.0

Valid

Total 19 100.0 100.0

H3_TrainingAfterVaccine



2 10.5 10.5 10.5

No 16 84.2 84.2 94.7

Yes 1 5.3 5.3 100.0

Valid

Total 19 100.0 100.0

H3_IfYesHowLongAfter



18 94.7 94.7 94.7

4 weeks 1 5.3 5.3 100.0

Valid

Total 19 100.0 100.0


51

H3_WhoConducted



7 36.8 36.8 36.8

CMO 1 5.3 5.3 42.1

DHO 1 5.3 5.3 47.4

District 1 5.3 5.3 52.6

MO 5 26.3 26.3 78.9

MO & DHO 3 15.8 15.8 94.7

No one 1 5.3 5.3 100.0

Valid

Total 19 100.0 100.0

H4_TrainingImprove



Don't know 2 10.5 10.5 10.5

No 10 52.6 52.6 63.2

Yes 2 10.5 10.5 73.7

Yes, 2-4 days would be better for more detailed information whether really an AEFI or not, whether to use adrenaline or not

1 5.3 5.3 78.9

Yes, refresher courses/training 1 5.3 5.3 84.2

Yes, type of vaccine was explained but forgot. More on vaccine component.

1 5.3 5.3 89.5

Yes, would be good if all health workers could get the training

1 5.3 5.3 94.7

Yes, would be good if others in BHU could be trained

1 5.3 5.3 100.0

Valid

Total 19 100.0 100.0


52

H5_Materials



2 10.5 10.5 10.5

Yes 17 89.5 89.5 100.0

Valid

Total 19 100.0 100.0

H6_Satisfied



1 5.3 5.3 5.3

No, large group of participants 1 5.3 5.3 10.5

Yes 17 89.5 89.5 100.0

Valid

Total 19 100.0 100.0

H7_Consent



2 10.5 10.5 10.5

No 7 36.8 36.8 47.4

No consent form at clinic. Only at school

1 5.3 5.3 52.6

Yes 9 47.4 47.4 100.0

Valid

Total 19 100.0 100.0

H8_IfYesAnyIssue



12 63.2 63.2 63.2

No 7 36.8 36.8 100.0

Valid

Total 19 100.0 100.0


53

H9_MethodsRecall



1 5.3 5.3 5.3

ask school health 1 5.3 5.3 10.5

Awareness before vaccination, during round 1 & 2 to complete

1 5.3 5.3 15.8

Call each due month for contact 1 5.3 5.3 21.1

Call teacher in school to find girl; use mobile phones of parent

1 5.3 5.3 26.3

Card 4 21.1 21.1 47.4

Dates on card, register, informed on advance, school health teachers

1 5.3 5.3 52.6

Information during meeting 1st and 2nd round

1 5.3 5.3 57.9

Information on importance completing during meeting at 1st and 2nd dose vaccination

1 5.3 5.3 63.2

Informed before and during 1st and 2nd rounds

1 5.3 5.3 68.4

Informed during 1st round to complete all the doses

1 5.3 5.3 73.7

Informed during vaccination, noted down phone number

1 5.3 5.3 78.9

Informed through meeting during 1st and 2nd dose vaccination

1 5.3 5.3 84.2

MCH register 1 5.3 5.3 89.5

Register 1 5.3 5.3 94.7

Vaccination card 1 5.3 5.3 100.0

Valid

Total 19 100.0 100.0


54

H10_Pop



10 1 5.3 8.3 8.3

16 1 5.3 8.3 16.7

23 1 5.3 8.3 25.0

35 1 5.3 8.3 33.3

40 1 5.3 8.3 41.7

120 1 5.3 8.3 50.0

197 1 5.3 8.3 58.3

308 1 5.3 8.3 66.7

310 1 5.3 8.3 75.0

492 1 5.3 8.3 83.3

621 1 5.3 8.3 91.7

1110 1 5.3 8.3 100.0

Valid

Total 12 63.2 100.0


Total 19 100.0


55

H11_Formula



6 31.6 31.6 31.6

Did not calculate, but send the reports to DHO

1 5.3 5.3 36.8

End of year, calculate coverage 1 5.3 5.3 42.1

MR/LB * 100 1 5.3 5.3 47.4

N/A 1 5.3 5.3 52.6

Not calculated 1 5.3 5.3 57.9

Not calculated. Calculate at district level

1 5.3 5.3 63.2

Standard formula 1 5.3 5.3 68.4

Total immunized / Total target * 100

5 26.3 26.3 94.7

Vaccine given / Target * 100 1 5.3 5.3 100.0

Valid

Total 19 100.0 100.0

H12_Rubella

Frequency Percentage



56

H13_FirstDose



96 1 5.3 6.3 6.3

97 2 10.5 12.5 18.8

98 1 5.3 6.3 25.0

98 1 5.3 6.3 31.3

99 1 5.3 6.3 37.5

100 7 36.8 43.8 81.3

100 1 5.3 6.3 87.5

105 1 5.3 6.3 93.8

111 1 5.3 6.3 100.0

Valid

Total 16 84.2 100.0


Total 19 100.0

H13_SecondDose



97 2 10.5 12.5 12.5

98 2 10.5 12.5 25.0

99 1 5.3 6.3 31.3

99 2 10.5 12.5 43.8

100 1 5.3 6.3 50.0

100 4 21.1 25.0 75.0

100 1 5.3 6.3 81.3

101 1 5.3 6.3 87.5

102 1 5.3 6.3 93.8

105 1 5.3 6.3 100.0

Valid

Total 16 84.2 100.0


Total 19 100.0


57

H13_ThirdDose



95 1 5.3 6.3 6.3

97 1 5.3 6.3 12.5

97 3 15.8 18.8 31.3

97 1 5.3 6.3 37.5

98 1 5.3 6.3 43.8

99 1 5.3 6.3 50.0

99 1 5.3 6.3 56.3

99 1 5.3 6.3 62.5

99 1 5.3 6.3 68.8

100 1 5.3 6.3 75.0

100 3 15.8 18.8 93.8

102 1 5.3 6.3 100.0

Valid

Total 16 84.2 100.0


Total 19 100.0

H14_DateReporting



2 10.5 10.5 10.5

After every round 1 5.3 5.3 15.8

Monthly 7 36.8 36.8 52.6

Monthly; after each HPV dose report

1 5.3 5.3 57.9

Often each dose round 1 5.3 5.3 63.2

Routine (Monthly), HPV (after campaign)

7 36.8 36.8 100.0

Valid

Total 19 100.0 100.0


58

H15_RegistryHPV



1 5.3 5.3 5.3

Health card, tally sheets 1 5.3 5.3 10.5

No 4 21.1 21.1 31.6

No update 1 5.3 5.3 36.8

No, separate register/form 7 36.8 36.8 73.7

Vaccine registry/logbook, health card, tally sheets

1 5.3 5.3 78.9

Vaccine registry/logbook, health card, vaccine stock forms

1 5.3 5.3 84.2

Yes, vaccine registry/logbook 1 5.3 5.3 89.5

Yes, Vaccine registry/logbook, health card, tally sheet, vaccine stock forms

2 10.5 10.5 100.0

Valid

Total 19 100.0 100.0

H16_DaysOutreach



1 5.3 5.3 5.3

7 outreach community 1 5.3 5.3 10.5

Monthly 10 52.6 52.6 63.2

Monthly ORC, 2 times BHU 1 5.3 5.3 68.4

One day per month 1 5.3 5.3 73.7

ORC - once in month; BHU - twice in month

1 5.3 5.3 78.9

ORC - one; BHU - for DPT/OVP/DT 1 5.3 5.3 84.2

ORC - one; Clinic - daily 1 5.3 5.3 89.5

ORC - twice month; BHU - once month

1 5.3 5.3 94.7

ORC - twice month; Clinic twice month in BHU

1 5.3 5.3 100.0

Valid

Total 19 100.0 100.0


59

H17_OutreachData



3 15.8 15.8 15.8

HH survey 1 5.3 5.3 21.1

MCH Register 1 5.3 5.3 26.3

MCH register, Talley sheet 1 5.3 5.3 31.6

Register 8 42.1 42.1 73.7

Register, Tally sheet 2 10.5 10.5 84.2

Register+note from ORC 1 5.3 5.3 89.5

Talley sheet used 1 5.3 5.3 94.7

VHW 1 5.3 5.3 100.0

Valid

Total 19 100.0 100.0

H18_ChangeOutreach



3 15.8 15.8 15.8

No changes required 15 78.9 78.9 94.7

Other, dates of different rounds

1 5.3 5.3 100.0

Valid

Total 19 100.0 100.0

H19_VacStored



1 5.3 5.3 5.3

Refrigerator 17 89.5 89.5 94.7

Refrigerator, Cold storage box

1 5.3 5.3 100.0

Valid

Total 19 100.0 100.0


60

H20_EquipOutreach



3 15.8 15.8 15.8

Cold boxes 1 5.3 5.3 21.1

Vaccine carrier 15 78.9 78.9 100.0

Valid

Total 19 100.0 100.0

H21_PowerCut



1 5.3 5.3 5.3

Cold box 2 10.5 10.5 15.8

Cold box, Kerosene model 1 5.3 5.3 21.1

Kerosene used 2 10.5 10.5 31.6

No 2 10.5 10.5 42.1

No major problem 1 5.3 5.3 47.4

No problem with electricity 1 5.3 5.3 52.6

Sometime, twice per month not more than 1 day

1 5.3 5.3 57.9

Sometimes for a few minutes

1 5.3 5.3 63.2

Switched on to Kerosene model

2 10.5 10.5 73.7

Transfer to cold box 2 10.5 10.5 84.2

Used generator 1 5.3 5.3 89.5

Used kerosene 1 5.3 5.3 94.7

Used kerosene refrigerator 1 5.3 5.3 100.0

Valid

Total 19 100.0 100.0


61

H22_ChangesColdChain



3 15.8 15.8 15.8

Got more vaccine carriers, cold boxes, ice packs

1 5.3 5.3 21.1

No 13 68.4 68.4 89.5

No changes for routine, special for campaign.

1 5.3 5.3 94.7

Requested more vaccine carriers, ice packs, cold boxes, and distributed to all BHUs

1 5.3 5.3 100.0

Valid

Total 19 100.0 100.0

H23_ProblemColdChain



1 5.3 5.3 5.3

No Problem 18 94.7 94.7 100.0

Valid

Total 19 100.0 100.0

H24_VMPolicy



3 15.8 15.8 15.8

No 2 10.5 10.5 26.3

Yes 14 73.7 73.7 100.0

Valid

Total 19 100.0 100.0


62

H24_IfYesUpdated



15 78.9 78.9 78.9

No update required 1 5.3 5.3 84.2

Old EPI manual 1 5.3 5.3 89.5

Separate guideline for HPV 1 5.3 5.3 94.7

Separate HPV manual 1 5.3 5.3 100.0

Valid

Total 19 100.0 100.0

H25_VacForcast



1 5.3 5.3 5.3

As per consumption/school record 1 5.3 5.3 10.5

Based on no. of ORC sessions, 1 vial for each session

1 5.3 5.3 15.8

Based on past quarters' use 1 5.3 5.3 21.1

Based on target population 5 26.3 26.3 47.4

Before - past experience; Present - formula as per manual

1 5.3 5.3 52.6

Consumption - no specific formula 1 5.3 5.3 57.9

Consumption basis 1 5.3 5.3 63.2

Health officer estimated 1 5.3 5.3 68.4

Household/school information 1 5.3 5.3 73.7

Max-AMC*4month for Max stock; Min-AMC*0.5 for Min stock

1 5.3 5.3 78.9

Minimum and maximum 1 5.3 5.3 84.2

Minimum and maximum stock 1 5.3 5.3 89.5

Use target population 2 10.5 10.5 100.0

Valid

Total 19 100.0 100.0


63

H26_ChangEstimate



6 31.6 31.6 31.6

No 12 63.2 63.2 94.7

Yes, non-enumerated girls came for the vaccination

1 5.3 5.3 100.0

Valid

Total 19 100.0 100.0

H27_Order



3 15.8 15.8 15.8

BHU 1 5.3 5.3 21.1

BHW 1 5.3 5.3 26.3

Both staff 1 5.3 5.3 31.6

CHU I/C 1 5.3 5.3 36.8

CHU staff 1 5.3 5.3 42.1

HA 3 15.8 15.8 57.9

Incharge 1 5.3 5.3 63.2

Incharge MCH 1 5.3 5.3 68.4

Indent 1 5.3 5.3 73.7

Indent form 1 5.3 5.3 78.9

Indent staff 1 5.3 5.3 84.2

MO incharge 1 5.3 5.3 89.5

Self 1 5.3 5.3 94.7

Sr HA, indent book 1 5.3 5.3 100.0

Valid

Total 19 100.0 100.0


64

H27_Delivery



4 21.1 21.1 21.1

Monthly 9 47.4 47.4 68.4

Quarterly 5 26.3 26.3 94.7

Quarterly to hospital and monthly to BHU

1 5.3 5.3 100.0

Valid

Total 19 100.0 100.0

H28_VacExpire



2 10.5 10.5 10.5

No 8 42.1 42.1 52.6

Yes 4 21.1 21.1 73.7

Yes, some OPV 1 5.3 5.3 78.9

Yes, TT vaccine 1 5.3 5.3 84.2

Yes; BCG-1 vial, OPV-2 vials 1 5.3 5.3 89.5

Yes; BCG-560 doses, OPV-1020 doses

1 5.3 5.3 94.7

Yes; OPV-27 vials, TT-83 vials 1 5.3 5.3 100.0

Valid

Total 19 100.0 100.0


65

H28_Action



14 73.7 73.7 73.7

Burn 1 5.3 5.3 78.9

Discarded 1 5.3 5.3 84.2

Disposed 1 5.3 5.3 89.5

Form filled and sent to DUED

1 5.3 5.3 94.7

Informed DVO 1 5.3 5.3 100.0

Valid

Total 19 100.0 100.0

H29_VVM



4 21.1 21.1 21.1

No 14 73.7 73.7 94.7

Yes, disposed off 1 5.3 5.3 100.0

Valid

Total 19 100.0 100.0


66

H30_VacShortage



2 10.5 10.5 10.5

No 11 57.9 57.9 68.4

Yes (DPT-HepB for 1 month in Aug 2010)

1 5.3 5.3 73.7

Yes (DPT-HepB for 1 month) 1 5.3 5.3 78.9

Yes (DPT-HepB for 1 week due to stock out at regional store)

1 5.3 5.3 84.2

Yes (DPT-HepB for 1 week in 2010)

1 5.3 5.3 89.5

Yes (DPT-HepB for 1.5 month due to stock out at regional store)

1 5.3 5.3 94.7

Yes (DPT-HepB for 3 weeks in 2010)

1 5.3 5.3 100.0

Valid

Total 19 100.0 100.0

H31_Injection



2 10.5 10.5 10.5

No 2 10.5 10.5 21.1

Yes 15 78.9 78.9 100.0

Valid

Total 19 100.0 100.0

H31_Stock



6 31.6 31.6 31.6

Yes 13 68.4 68.4 100.0

Valid

Total 19 100.0 100.0


67

H32_ChangeWasteDisposal



1 5.3 5.3 5.3

No 15 78.9 78.9 84.2

Yes 2 10.5 10.5 94.7

Yes, autoclave and burnt 1 5.3 5.3 100.0

Valid

Total 19 100.0 100.0

H33_ProblemWaste



1 5.3 5.3 5.3

No 16 84.2 84.2 89.5

Yes, no autoclave 1 5.3 5.3 94.7

Yes, pit with no roof 1 5.3 5.3 100.0

Valid

Total 19 100.0 100.0


68

H34_FormulaWaste



8 42.1 42.1 42.1

(B+D+E)-C/B+D+E * 100 1 5.3 5.3 47.4

(Total doses used - total immunized)/Total doses used * 100

2 10.5 10.5 57.9

As per the formula provided 1 5.3 5.3 63.2

Doses used+doses discarded (expiry & frozen) - doses used/doses used+doses discarded * 100

2 10.5 10.5 73.7

No of doses used / No of doses issued * 100

1 5.3 5.3 78.9

No vaccine used - no immunized / Total vaccine used * 100

1 5.3 5.3 84.2

Standard formula 2 10.5 10.5 94.7

Vaccine wastage not calculated. Calculated at national level

1 5.3 5.3 100.0

Valid

Total 19 100.0 100.0

H35_WastageRate



4 21.1 21.1 21.1

0 13 68.4 68.4 89.5

First round - 3 vials, Second round - 4 vials

1 5.3 5.3 94.7

NA 1 5.3 5.3 100.0

Valid

Total 19 100.0 100.0


69

H36_ChangetoReduce



7 36.8 36.8 36.8

Fix the dates 1 5.3 5.3 42.1

No 11 57.9 57.9 100.0

Valid

Total 19 100.0 100.0

H37_SupVisit



1 5.3 5.3 5.3

1 visit, written report kept at district level

1 5.3 5.3 10.5

2 times 2 10.5 10.5 21.1

2 times, no written visit report 2 10.5 10.5 31.6

2-3 times 2 10.5 10.5 42.1

3 times, written report available 1 5.3 5.3 47.4

4 times, (1 written report) 1 5.3 5.3 52.6

All 3 rounds during HPV campaign 2 10.5 10.5 63.2

Daily, person near by for campaign, not during routine

1 5.3 5.3 68.4

Frequently by DHO/others 1 5.3 5.3 73.7

No report, central level supervisor in Feb 2011

1 5.3 5.3 78.9

Once, no written visit report 2 10.5 10.5 89.5

Quarterly, no written report 1 5.3 5.3 94.7

Twice in a year, written report available

1 5.3 5.3 100.0

Valid

Total 19 100.0 100.0


70

H38_WhoVisited



5 26.3 26.3 26.3

DHO 4 21.1 21.1 47.4

DHO, DG 1 5.3 5.3 52.6

DHO, other officials 1 5.3 5.3 57.9

DHO/ADHO 2 10.5 10.5 68.4

DMO/DHO 1 5.3 5.3 73.7

HPV Supervisor 1 5.3 5.3 78.9

MO, DHO 1 5.3 5.3 84.2

MOH staff 1 5.3 5.3 89.5

National supervisor 1 5.3 5.3 94.7

RH-PO & WHO 1 5.3 5.3 100.0

Valid

Total 19 100.0 100.0

H39_AEFI



3 15.8 15.8 15.8

Yes 16 84.2 84.2 100.0

Valid

Total 19 100.0 100.0

H40_ChangeProtocal



3 15.8 15.8 15.8

NA 1 5.3 5.3 21.1

No 14 73.7 73.7 94.7

Yes, complain form 1 5.3 5.3 100.0

Valid

Total 19 100.0 100.0


71

H41_AEFIReport



2 10.5 10.5 10.5

No 15 78.9 78.9 89.5

No (1 Abscess due to DPT-HepB, treated in Hospital)

1 5.3 5.3 94.7

Yes, 1 (headache) 1 5.3 5.3 100.0

Valid

Total 19 100.0 100.0

H42_LunchHPV



1 5.3 5.3 5.3

No 15 78.9 78.9 84.2

No (Principal for lunching) 1 5.3 5.3 89.5

Yes 2 10.5 10.5 100.0

Valid

Total 19 100.0 100.0

H43_HEMaterials



1 5.3 5.3 5.3

Health education sessions, public meeting

1 5.3 5.3 10.5

Posters 1 5.3 5.3 15.8

Posters, Brochures 1 5.3 5.3 21.1

Posters, brochures, health education sessions, public meetings

8 42.1 42.1 63.2

Posters, health education sessions, public meetings

5 26.3 26.3 89.5

Public meetings 2 10.5 10.5 100.0

Valid

Total 19 100.0 100.0


72

H44_Resistance



2 10.5 10.5 10.5

No 14 73.7 73.7 84.2

Yes, 1 girl (who later got the vaccine)

1 5.3 5.3 89.5

Yes, only on the 1st day of 1st round when consent form was required

1 5.3 5.3 94.7

Yes, people from town came after the 1st round

1 5.3 5.3 100.0

Valid

Total 19 100.0 100.0

H45_MediaFocus



2 10.5 10.5 10.5

No 2 10.5 10.5 21.1

Yes 1 5.3 5.3 26.3

Yes; Radio 4 21.1 21.1 47.4

Yes; Radio & Newspaper 1 5.3 5.3 52.6

Yes; Radio, TV & Newspaper 5 26.3 26.3 78.9

Yes; TV & Newspaper 2 10.5 10.5 89.5

Yes; TV & Radio 1 5.3 5.3 94.7

Yes; TV, Radio & Official info 1 5.3 5.3 100.0

Valid

Total 19 100.0 100.0


73

H46_HPVSchedule



2 10.5 10.5 10.5

0 day, after 2 months and after 6 months of 1st dose

4 21.1 21.1 31.6

0 day, after 2 months of 1st dose and after 4 months of 2nd dose

4 21.1 21.1 52.6

0,2,6 5 26.3 26.3 78.9

0,2,6 months 1 5.3 5.3 84.2

Integrated to regular EPI 1 5.3 5.3 89.5

Regular schedule 1 5.3 5.3 94.7

Schedule not clear 1 5.3 5.3 100.0

Valid

Total 19 100.0 100.0

H47_Disease



1 5.3 5.3 5.3

CaCx 17 89.5 89.5 94.7

Cervical cancer + genital warts

1 5.3 5.3 100.0

Valid

Total 19 100.0 100.0


74

H48_Information



2 10.5 10.5 10.5

Benefits to the girl, vaccine schedule/when to return, normal side effects

1 5.3 5.3 15.8

Benefits to the girl, vaccine schedule/when to return, normal side effects, what side effects they should return for

1 5.3 5.3 21.1

Benefits to the girl, vaccine schedule/when to return, what side effects they should return for

1 5.3 5.3 26.3

Disease it protects against, benefits to the girl, vaccine schedule/when to return, normal side effects

1 5.3 5.3 31.6

Diseases it protects against, vaccine schedule/when to return, what side effects they should return for, other health message

1 5.3 5.3 36.8

Diseases it protects against, benefits to the girl, vaccine schedule/when to return, what side effects they should return for, bring vaccination card, other health message

1 5.3 5.3 42.1

Name of the vaccine, benefits to the girl, vaccine schedule/when to return, normal side effects

1 5.3 5.3 47.4

Name of the vaccine, Diseases it protects against, benefits to the girl, vaccine schedule/when to return, normal side effects, bring vaccination card

3 15.8 15.8 63.2

Name of the vaccine, Diseases it protects against, benefits to the girl, vaccine schedule/when to return, normal side effects, risk of synkopy, what side effects they should return for, bring vaccination card

1 5.3 5.3 68.4

Valid

Name of the vaccine, Diseases it 2 10.5 10.5 78.9


75



protects against, benefits to the girl, vaccine schedule/when to return, what side effects they should return for, bring vaccination card, other health message

Name of the vaccine, Diseases it protects against, vaccine schedule/when to return, normal side effects

1 5.3 5.3 84.2

Name of the vaccine, Diseases it protects against, vaccine schedule/when to return, normal side effects, bring vaccination card

2 10.5 10.5 94.7

Name of the vaccine, Diseases it protects against, vaccine schedule/when to return, what side effects they should return for

1 5.3 5.3 100.0

Total 19 100.0 100.0

H49_ColdChain



5 26.3 26.3 26.3

Don't know 1 5.3 5.3 31.6

No 13 68.4 68.4 100.0

Valid

Total 19 100.0 100.0

H49_VacTrans



6 31.6 31.6 31.6

No 13 68.4 68.4 100.0

Valid

Total 19 100.0 100.0


76

H49_Wastage



6 31.6 31.6 31.6

NA 1 5.3 5.3 36.8

No 12 63.2 63.2 100.0

Valid

Total 19 100.0 100.0

H49_ComMaterial



7 36.8 36.8 36.8

No 12 63.2 63.2 100.0

Valid

Total 19 100.0 100.0

H49_Training



6 31.6 31.6 31.6

No 13 68.4 68.4 100.0

Valid

Total 19 100.0 100.0

H49_Other



7 36.8 36.8 36.8

No 12 63.2 63.2 100.0

Valid

Total 19 100.0 100.0


77

H50_Effect



2 10.5 10.5 10.5

Improved the EPI programme 1 5.3 5.3 15.8

No 1 5.3 5.3 21.1

No effect 15 78.9 78.9 100.0

Valid

Total 19 100.0 100.0

H51_Process



2 10.5 10.5 10.5

Generally smooth - minor problem

2 10.5 10.5 21.1

Very smooth - no problem 15 78.9 78.9 100.0

Valid

Total 19 100.0 100.0


78

H52_Advice



6 31.6 31.6 31.6

Advocacy, involvement of media, benefits of the vaccine to be explained properly.

1 5.3 5.3 36.8

Already have connection MOE+MOH

1 5.3 5.3 42.1

Awareness among people, training, supervision

1 5.3 5.3 47.4

Awareness and advantages of the vaccine, collaboration from school & community

1 5.3 5.3 52.6

Awareness at all levels. Explain vaccine safety and its benefit, and it is free of cost.

1 5.3 5.3 57.9

Awareness to community, HW to be confident, prepare for AEFI management

1 5.3 5.3 63.2

Awareness, cooperation from all relevant stakeholders, health educators

1 5.3 5.3 68.4

Create adequate awareness of HPV vaccine and the problems of CaCx

1 5.3 5.3 73.7

Explain the problem of CaCx to public and its prevention. Intersectoral coordination funds

1 5.3 5.3 78.9

NA 1 5.3 5.3 84.2

Need to sensitize the population, explain importance, address rumors, village leaders

1 5.3 5.3 89.5

No major AEFI, no major cold chain 1 5.3 5.3 94.7

Plan, more space, post observation, training space

1 5.3 5.3 100.0

Valid

Total 19 100.0 100.0


79

H53_Reconstituted



13 68.4 68.4 68.4

Yes 6 31.6 31.6 100.0

Valid

Total 19 100.0 100.0

H54_VacStored



13 68.4 68.4 68.4

Yes 6 31.6 31.6 100.0

Valid

Total 19 100.0 100.0

H55_Administration



13 68.4 68.4 68.4

No vaccination today 1 5.3 5.3 73.7

Not observed 4 21.1 21.1 94.7

Yes 1 5.3 5.3 100.0

Valid

Total 19 100.0 100.0

H56_ADSyringes



13 68.4 68.4 68.4

Yes 6 31.6 31.6 100.0

Valid

Total 19 100.0 100.0


80

H57_Recapped



13 68.4 68.4 68.4

No 6 31.6 31.6 100.0

Valid

Total 19 100.0 100.0

H58_Disposed



14 73.7 73.7 73.7

Yes 5 26.3 26.3 100.0

Valid

Total 19 100.0 100.0

H59_DateOpened



14 73.7 73.7 73.7

Yes 5 26.3 26.3 100.0

Valid

Total 19 100.0 100.0

H59_VialDiscarded



14 73.7 73.7 73.7

No, except BCG/MR 1 5.3 5.3 78.9

Yes 4 21.1 21.1 100.0

Valid

Total 19 100.0 100.0


81

H59_Observation



H60_NumUnsafe



H61_GirlsObserved



14 73.7 73.7 73.7

Yes 4 21.1 21.1 94.7

Yes, half-an hour 1 5.3 5.3 100.0

Valid

Total 19 100.0 100.0

H62_Refrigerator



1 5.3 5.3 5.3

Yes 18 94.7 94.7 100.0

Valid

Total 19 100.0 100.0

H63_ThermoOutside



1 5.3 5.3 5.3

No 5 26.3 26.3 31.6

Yes 13 68.4 68.4 100.0

Valid

Total 19 100.0 100.0


82

H64_ThermoInside



1 5.3 5.3 5.3

Yes 18 94.7 94.7 100.0

Valid

Total 19 100.0 100.0

H65_Temp



1 5.3 5.3 5.3

No, 9dC (power failed for half an hour)

1 5.3 5.3 10.5

Yes 5 26.3 26.3 36.8

Yes, 2dC 2 10.5 10.5 47.4

Yes, 3dC 3 15.8 15.8 63.2

Yes, 4dC 4 21.1 21.1 84.2

Yes, 5dC 2 10.5 10.5 94.7

Yes, 6dC 1 5.3 5.3 100.0

Valid

Total 19 100.0 100.0

H66_Log



2 10.5 10.5 10.5

Yes 17 89.5 89.5 100.0

Valid

Total 19 100.0 100.0


83

H67_TempRecord



1 5.3 5.3 5.3

Daily 3 15.8 15.8 21.1

Twice daily 15 78.9 78.9 100.0

Valid

Total 19 100.0 100.0

H68_WeekedRecord



3 15.8 15.8 15.8

No (if vaccine required on holiday - done, and not done for only 1 day holiday)

1 5.3 5.3 21.1

Yes 15 78.9 78.9 100.0

Valid

Total 19 100.0 100.0

H69_FEFO



2 10.5 10.5 10.5

Yes 17 89.5 89.5 100.0

Valid

Total 19 100.0 100.0

H70_VacExpire



1 5.3 5.3 5.3

No 14 73.7 73.7 78.9

Yes 4 21.1 21.1 100.0

Valid

Total 19 100.0 100.0


84

H70_IfYesVacNum



16 84.2 84.2 84.2

As noted before 1 5.3 5.3 89.5

OPV-1 vial 1 5.3 5.3 94.7

OPV-27 vials and TT-83 vials 1 5.3 5.3 100.0

Valid

Total 19 100.0 100.0

H71_VVM



1 5.3 5.3 5.3

Yes, all vaccine usable 17 89.5 89.5 94.7

Yes, all vaccine usable (except expired one)

1 5.3 5.3 100.0

Valid

Total 19 100.0 100.0

H72_VacVVM



1 5.3 5.3 5.3

NA 1 5.3 5.3 10.5

Not applicable, no stage 2 12 63.2 63.2 73.7

Yes 5 26.3 26.3 100.0

Valid

Total 19 100.0 100.0

H73_AirCirculation



2 10.5 10.5 10.5

Yes 17 89.5 89.5 100.0

Valid

Total 19 100.0 100.0


85

H74_Poster



3 15.8 15.8 15.8

Yes 16 84.2 84.2 100.0

Valid

Total 19 100.0 100.0

H75_AdqSpace



4 21.1 21.1 21.1

Yes 15 78.9 78.9 100.0

Valid

Total 19 100.0 100.0

H75_Clean



17 89.5 89.5 89.5

Yes 2 10.5 10.5 100.0

Valid

Total 19 100.0 100.0

H75_Organized



17 89.5 89.5 89.5

Yes 2 10.5 10.5 100.0

Valid

Total 19 100.0 100.0


86

H76_Dispose



2 10.5 10.5 10.5

Open bucket 1 5.3 5.3 15.8

Safety box 16 84.2 84.2 100.0

Valid

Total 19 100.0 100.0

H77_SafetyBox



3 15.8 15.8 15.8

Pit-burned 14 73.7 73.7 89.5

Pit-burned, Pit-buried 2 10.5 10.5 100.0

Valid

Total 19 100.0 100.0

H78_NeedlesObserved



4 21.1 21.1 21.1

No 15 78.9 78.9 100.0

Valid

Total 19 100.0 100.0

H79_WastSite



5 26.3 26.3 26.3

No 12 63.2 63.2 89.5

Yes 2 10.5 10.5 100.0

Valid

Total 19 100.0 100.0


87

H80_Observation



16 84.2 84.2 84.2

Open pit 1 5.3 5.3 89.5

Separate disposal pit for vials and sharps

1 5.3 5.3 94.7

Well maintained 1 5.3 5.3 100.0

Valid

Total 19 100.0 100.0

Frenquency tables for interview with girls (please refer to annex 4 for the data dictionary and the exact questionnaire)

District



Jigmecholing 5 11.1 11.1 11.1

Pondkllha 1 2.2 2.2 13.3

Punakha 2 4.4 4.4 17.8

SARPANG 5 11.1 11.1 28.9


Trongsa (T/Phy) 5 11.1 11.1 55.6

Tsirang 9 20.0 20.0 75.6

WANGDUE 6 13.3 13.3 88.9

Yebitaplsa Hospital 5 11.1 11.1 100.0

Valid

Total 45 100.0 100.0


88

G1_HealthCard



1 2.2 2.2 2.2

N/A 1 2.2 2.2 4.4

No 10 22.2 22.2 26.7

Yes 33 73.3 73.3 100.0

Valid

Total 45 100.0 100.0

G1_VacToday



7 15.6 15.6 15.6

HPV 38 84.4 84.4 100.0

Valid

Total 45 100.0 100.0

G1_CardUpdated



20 44.4 44.4 44.4

Old health card - not updated to include new vaccine

24 53.3 53.3 97.8

Separate card for HPV 1 2.2 2.2 100.0

Valid

Total 45 100.0 100.0

G2_VacReceived



42 93.3 93.3 93.3

Does not know 2 4.4 4.4 97.8

N/A 1 2.2 2.2 100.0

Valid

Total 45 100.0 100.0


89

G3_KnowHPV



1 2.2 2.2 2.2

No 4 8.9 8.9 11.1

Yes 40 88.9 88.9 100.0

Valid

Total 45 100.0 100.0

G3_IfYesPrevent



4 8.9 8.9 8.9

Answer correct 32 71.1 71.1 80.0

Answer incorrect 7 15.6 15.6 95.6

Does not know 2 4.4 4.4 100.0

Valid

Total 45 100.0 100.0

G4_Message



BBS, teacher 1 2.2 2.2 2.2

BHU staff 1 2.2 2.2 4.4

BHU staff, teacher 4 8.9 8.9 13.3

HA 2 4.4 4.4 17.8

HCW 1 2.2 2.2 20.0

HCW, friend, public meeting 1 2.2 2.2 22.2

HCW, Teacher 1 2.2 2.2 24.4

Health coordinator, BBS, Newspaper 1 2.2 2.2 26.7

Health coordinator, principal 1 2.2 2.2 28.9

Health educator, teacher, radio, TV 1 2.2 2.2 31.1

Valid

Health worker, BBS, teacher 1 2.2 2.2 33.3


90



Hospital staff, school 1 2.2 2.2 35.6

Hospital staff, teachers, friends 3 6.7 6.7 42.2

Hospital staff, teachers, parents 1 2.2 2.2 44.4

Newspaper, teacher 1 2.2 2.2 46.7

Newspaper, TV, HCW 1 2.2 2.2 48.9

Newspaper, TV, HCW, teacher 1 2.2 2.2 51.1

Parents 1 2.2 2.2 53.3

Parents, TV 1 2.2 2.2 55.6

School 1 2.2 2.2 57.8

School health incharge 1 2.2 2.2 60.0

School health incharge, Friends 1 2.2 2.2 62.2

School health incharge, health campaign

1 2.2 2.2 64.4

School teacher, BBS 1 2.2 2.2 66.7

School, BAU staff 1 2.2 2.2 68.9

School, BHU staff 2 4.4 4.4 73.3

School, hospital staff 1 2.2 2.2 75.6

School, TV, Poster 1 2.2 2.2 77.8

Teacher 2 4.4 4.4 82.2

Teacher, health minister's message 1 2.2 2.2 84.4

Teacher, Radio, TV 1 2.2 2.2 86.7

Teacher, TV 1 2.2 2.2 88.9

TV 1 2.2 2.2 91.1

TV, friend 1 2.2 2.2 93.3

TV, friends 1 2.2 2.2 95.6

TV, friends, teacher 1 2.2 2.2 97.8

TV, School Health coordinator, Friends

1 2.2 2.2 100.0

Total 45 100.0 100.0


91

G5_NextVaccination



4 8.9 8.9 8.9

No 1 2.2 2.2 11.1

Yes - answer correct 38 84.4 84.4 95.6

Yes - answer incorrect 2 4.4 4.4 100.0

Valid

Total 45 100.0 100.0

G6_Reaction



7 15.6 15.6 15.6

No 8 17.8 17.8 33.3

Yes - answer correct 25 55.6 55.6 88.9

Yes - answer incorrect 5 11.1 11.1 100.0

Valid

Total 45 100.0 100.0


92

G7_Comments



6 13.3 13.3 13.3

Benefits all women 1 2.2 2.2 15.6

Explain detail HPV purposes and benefits

1 2.2 2.2 17.8

Good to get the vaccine which prevents CaCx

1 2.2 2.2 20.0

Important 1 2.2 2.2 22.2

Information to have vaccination to prevent cervical cancer

1 2.2 2.2 24.4

It benefits to all women to prevent from cervical cancer

1 2.2 2.2 26.7

It should be given to all women 1 2.2 2.2 28.9

It should continue 1 2.2 2.2 31.1

Need more awareness 1 2.2 2.2 33.3

No comments 21 46.7 46.7 80.0

Saving life 1 2.2 2.2 82.2

Saving life from this dying disease 1 2.2 2.2 84.4

Should be given to all children 2 4.4 4.4 88.9

Thanks for government for giving this vaccine free of cost to prevent cervical cancer

2 4.4 4.4 93.3

Thanks for government for giving this vaccine to prevent cervical cancer

2 4.4 4.4 97.8

To continue providing this vaccine to all girls

1 2.2 2.2 100.0

Valid

Total 45 100.0 100.0


93

Annex 4

HPV Vaccine Post-Introduction Evaluation (PIE) tool

HPV Questionnaires

Each question in the questionnaire is accompanied by an abbreviation, explained below.

Abbreviation Explanation Comment

GEN Generic PIE questions should be included in all evaluations

CENT Questions to be asked at the central level only

These questions are shaded in grey

DIST Questions to be asked at the district level only

Questionnaire 1.1 Central/District

Questionnaire 1.2 Health Facility

Questionnaire 1.3 Interview with girls


94

Questionnaire 1.1 — CENTRAL/DISTRICT

OBLIGATORY HPV Vaccination

Date of interview: ________________ Name of interviewer: ______________________________

This questionnaire was conducted at: (insert name of country, region or district)

Central level: _____________________

District level: _____________________

Name(s) and title(s) of person(s) interviewed (please list all persons that you interviewed):

EPI manager/person responsible for vaccinations (or their deputy) should be interviewed

Name: ________________________ Title: _______________________

Name: ________________________ Title: _______________________

Name :________________________ Title: _______________________

Contact details of most senior person:

Telephone: __________________ E-mail address: _______________________

Name of new vaccine(s) being evaluated: _______________________

New vaccine preparation: (e.g. fully liquid, liquid lyophilized, manufacturer)

_____________________________________________________________

New vaccine presentation: (e.g. prefilled syringe, 1-dose vial, 2-dose vial)

______________________________________________________________


95

Documents to request at beginning of interview:

Document/data Document received

Document reported to exist but not

available at time of interview

Document unavailable

Copy of national immunization schedule (central level only)

Introduction plan for new vaccine

Training materials/reference documents utilized at new vaccine training

Vaccine management guidelines

Media campaign/social mobilization/education materials (e.g. brochures, posters, pamphlets)

Vaccine stock records

Supervisor's book/site-visit reports (regional and district level only)

Injection safety/waste-management policy document

Wastage reports

AEFI protocol/reporting form

AEFI logbook/registry

National coverage and drop-out rates (central level)


96

Abbreviation Background information Central/Regional/District Questionnaire

GEN 1. Date of obligatory HPV Vaccination was started at national/regional/district level

Note: If interviewing region or district, put date for appropriate area.

(DD/MM/YYYY) _____ / _____/ ______

CENT 2. Was the HPV Vaccine introduced nationwide or was it a phased introduction?

National introduction (all regions and districts at once)

Phased introduction (explain) _______________________

GEN 3. What is the population of girls 12 years of age in this country/region/ district?

Number of girls 12 years of age ______________

Source/Year____________

CENT 4. What factors influenced the decision for introduction of the HPV vaccine?

Check all that apply

Strong political will

Strong gynecologist/oncologist associations

Introduction by neighbouring countries

Disease burden data available nationally

Visit by international adviser

Other influences (specify)

CENT 5. Was the national immunization advisory committee supportive of the decision to introduce the new vaccine?

Yes No Don’t know

If no, what were their reasons:

_________________________

CENT 6. What is the current national immunization schedule?

Note: Ask for a copy of the schedule for all EPI vaccines (central level only).

Copy of schedule received Yes No

CENT 7. Was the immunization schedule changed when the new vaccine was introduced? If yes, why?

Yes No Don’t know

If yes, reason _______________________________


97

CENT 8. What is the schedule for the HPV Vaccine?

Insert age that dose is given

Schedule: Dose 1 ___________

Dose 2 ___________

Dose 3 ___________

CENT 9. Was HPV vaccination linked to any adolescent health package?

Yes No Don’t know

If yes, please describe ________________________________

CENT 10. Does MKD have a National Strategy for cervical cancer prevention and control?

Yes No Don’t know

If yes, please describe ________________________________

CENT 11. Has the cervical screening programme strategy been changed since introduction of HPV vaccine?

Yes No Don’t know

If yes or no, please describe _______________________________

GEN (optional)

12. Could you please tell what disease(s) does HPV Vaccine prevent?

Note: HPV cervical precancer and cancer caused by serotypes included into vaccine (around 70% of all). For quadrivalent vaccine – anogenital warts.

Pre-Implementation Planning and Vaccine Introduction Process

Central/District Questionnaire

GEN 13. Do you have a central/district HPV Vaccine introduction plan or timeline for introduction activities?

Note: For example, if someone from the district only has a national plan, just check national plan. If they have a national and a district plan check both.

Yes, national plan/timeline

Yes, district plan/timeline

Interviewer, please ask for a copy at time of interview. Review later to ensure essential components are included.

No. If no, why not?

______________________________


98

CENT Ask only if response to question 13 was "yes"

14. Did you receive support or use guidelines to develop your introduction plan/timeline?

Yes. If yes, specify support?

_______________________

No. If no, why not?

________________________

Don’t know

Training Central/District Questionnaire

GEN 15. Describe the training organized for your employees for the HPV Vaccine introduction, if any.

Target audience for the training

Doctors

Nurses

Health-care workers

Other (specify)

Type of training

Cascade

Region-by-region

Other (specify)

Was training conducted before vaccine introduction Yes No

If yes, how long before _________________

Was training conducted after vaccine introduction Yes No

If yes, how long after ____________________

How long was the training? ________________

Who conducted the training at each level?

Regions__________________________

District ___________________________

Health facilities _____________________

Other comments on training

_______________________

GEN 16. How were the trainings financed?


99

GEN 17. Do you think there are any ways in which the training could be improved for next time?

Yes No Don’t know

If yes, please describe

________________________________

GEN 18. What educational and reference materials were provided to participants at time of training? Ask for samples.

Vaccine Delivery Central/District Questionnaire

GEN 19. Is informed consent required? Yes No

GEN 20. If yes, does it pose any issues? Yes No

GEN 21. What methods were used for recall or to enhance completion of 3–dose schedule?

Please list the methods

Vaccine Coverage Central/District Questionnaire

GEN 22. Was the immunization database updated to accommodate information on the new vaccine?

Yes No Don’t know

GEN 23. What formula do you use to calculate vaccine coverage? Include the source of the numerator (doses administered) and denominator (target population).

Formula

Numerator source _____________________

Denominator source___________________

Correct formula used Yes No

GEN 24. What was rubella vaccine coverage in girls of 14 years of age in the year before the HPV vaccine introduction?

Note: Use year before new vaccine introduction or closest administrative period.

Rubella coverage _________ year _________

GEN 25. What is the coverage of the first and last dose of the HPV vaccine for the most recent administrative period?

HPV first dose coverage _________

HPV second dose coverage _________

HPV last dose coverage _________

Specify the period;


100

CENT 26. In the last year, what proportion of regions/districts/health facilities sent all monthly immunization summary forms completed and submitted on time?

Percentage of regions/districts/health facilities submitting reports on time every month ______

Percentage reports complete _________

(Of reports received, how many have all key information completed for every month)?

Cold-Chain Management Central/District Questionnaire

GEN 27. Discuss any changes you had to make in the cold chain before introduction of the HPV vaccine.

Note: Try to distinguish cold chain expansion/replacement of equipment that is part of normal cold-chain rehabilitation from changes made specifically to accommodate the new vaccine.

CENT 28. Were any problems with the cold chain identified after the introduction of the HPV vaccine? If yes, what were the problems and how have the problems been addressed?

No problems

Inadequate space

Frozen vaccine

Malfunctioning refrigerators

Power supply/fuel shortage

Other (specify)

CENT 29. Do you use freeze watch monitors during vaccine transportation?

Yes No Don’t know

Vaccine Management, Transport & Logistics


GEN 30. Do you have immunization policy guidelines for vaccine management? If yes, have they been updated to include the new vaccine? Please provide a copy at time of interview.

Yes No

CENT 31. How do you forecast vaccine requirements?

GEN 32. Did the estimated needs change with introduction of the HPV vaccine?

Yes No Don’t know

If yes, why? ___________________


101

CENT 33. How are vaccines ordered?

CENT 34. Please describe how vaccines are transported to the regions/districts/health facilities.

CENT 35. How often do you send out vaccine shipments and supplies from your level to the next level?

CENT 36. Did the frequency of deliveries change with introduction of the HPV vaccine? If yes, by how much?

Yes No Don’t know

If yes,

Frequency of delivery before introduction ______ times/year

Frequency of delivery after introduction ________ times/year

Reason for change?

CENT 37. Please describe how the transportation of vaccines to outreach sites has changed with the introduction of the HPV vaccine.

CENT 38. What effect did the HPV vaccine have on dry storage space requirements?

CENT 39. What were the costs associated with increased transport or cold-chain requirements?

Please state how many of the following were required:

Extra trucks/cars rental or purchase

_____________________

Extra logistic staff

__________________________________

Extra petrol

_______________________________________

Extra cold-chain space

_______________________________

Other costs (specify)

________________________________


102

CENT 40. Who paid for these extra costs?

CENT 41. Did you run out of any vaccines, including the HPV vaccine, or vaccine supplies in the past six months?

Yes, vaccines (specify) ___________

Yes, vaccine supplies (specify) ___________

No

If yes, how many weeks ___________

If yes, reason for stock out _____________

CENT 42. Have you had any vaccine expirations in the last six months? If yes, what did you do with the expired stock?

Yes No

If yes, action taken _______________

CENT 43. Have you had any vaccine with the vaccine vial monitor (VVM) in stage III or IV in the last six months? If yes, what did you do with these vaccines?

Yes No

If yes, action taken ____________

CENT 44. Are vaccine orders/deliveries tied to injection supplies (i.e. bundling)?

Note: Look at stock records to get this information.

Yes No

Verified by checking stock records

Yes No

Waste Management & Injection Safety


GEN 45. Describe the waste-disposal policy/plan at each level.

GEN 46. Does each level generally follow these guidelines?

Yes No Don’t know

GEN 47. Did you have to make changes to your guidance for your waste-disposal system for introduction of the HPV vaccine? If yes explain.

GEN 48. Did you have to make changes to your guidelines regarding injection safety for introduction of the HPV vaccine? If yes, explain.


103

Vaccine Wastage Central/District Questionnaire

GEN 49. What formula is used to calculate vaccine wastage and what is the source of the data.

Ask for wastage report.

Vaccine wastage not calculated

Formula:

Data source, numerator ______________________

Data source, denominator ____________________

Is provided formula correct? (See Table 10)

Yes No

Source of data:

Stock books

Summary sheets

Other

GEN 50. What is the vaccine wastage rate of the HPV vaccine?

Note: If vaccine wastage rate is unknown for new vaccine because PIE is done before administrative data are available, record anecdotal reports or attempt part-year calculation.

HPV vaccine wastage rate _____________%

GEN 51. Did you change anything about the way you administer vaccines, to reduce wastage of the HPV vaccine?

Monitoring and Supervision Central/Regional/District Questionnaire

GEN 52. How often are supervisory visits made to the regional/district/health-facility level?

Regional level ___________

District level ___________

Health-facility level ___________

GEN 53. Have you or a member of your staff or a partner organization made supervisory visits, to the districts/health facilities since HPV vaccine introduction? If so, how often and by whom?

Yes No

If yes, how often _____________

By whom ___________________

If no, why not? ________________


104

GEN 54. How do supervisors give feedback to sites visited?

Written

Supervisory logbook

Supervisory checklist

Send site visit report

Other (specify) ________

Oral

Discussion with staff

Other (specify) ________

GEN 55. What are the main issues that came up at the last two supervisory visits? Are they specifically related to introduction of the HPV vaccine? How have they been resolved?

a.

b.

c.

GEN 56. Are follow-up visits conducted at sites with inadequate performance and continuing problems?

Yes No

GEN 57. Have you received a supervisory visit? If yes, when and by whom?

Yes No

When _______________

By whom ________________________

Ask to see a copy of the visit report.

Adverse Events Following Immunization (AEFI)

Central/Regional/District Questionnaire

GEN 58. Do you have a system and written protocol for monitoring and reporting AEFIs for all vaccines? Please describe the procedure.

Ask for a copy of the AEFI protocol and reporting form.

Yes No

If no, why not _______________________

GEN 59. Do you have a crisis plan in place to manage AEFIs? Please describe.


105

GEN 60. Did you make any changes to the AEFI protocol specifically for the HPV vaccine?

GEN 61. Have you had any reported AEFIs for the HPV vaccine or another vaccine since the HPV vaccine was introduced?

Note: Verify using AEFI logbook/registry if available.

Yes No Don’t know

If yes,

How many for the HPV vaccine _________________________

How many for a traditional vaccine (specify) _______________

What were the AEFIs_________________________

How were they handled? _________________________

Advocacy & Communication Central/Regional/District Questionnaire

GEN 62. Did you have an official launch ceremony at the time of the HPV vaccine introduction?

Note: If yes, what did it involve, was it successful, did it get much media coverage, how long before the introduction of the HPV vaccine did it take place?

Yes No Don’t know

If yes, describe ________________________

If no, why not? ________________________

GEN 63. Did you use any media outlets to promote the HPV vaccine and inform/educate the community about the vaccine?

Note: Please ask for copies of any materials.

Check all that apply:

Radio

Television

Community groups

Town crier

Celebrity

Government officials

Other (specify)

Main messages ______________________________________


106

GEN 64. Did you prepare or distribute any health education material for the community on the HPV vaccine? If yes, what were they? Who were the target audiences? When and how were they distributed?

Note: Please ask for copies of any materials.

Check all that apply:

Posters

Brochures

Flyers

Clothing (t-shirts, hats etc.)

Other (specify)

Target audiences ____________________________________

Main messages _____________________________________

Sustainability Central/Regional Questionnaire

CENT 65. Is there a budget line for vaccine purchases in the national budget?

CENT 66. How are traditional EPI vaccines financed?

Note: List all sources that pay for the vaccine.

CENT 67. How is the HPV vaccine paid for?


CENT 68. How are the operational delivery costs of HPV vaccine paid for?


CENT 69. Do you plan to introduce any more new vaccines in the future? If yes, which one(s) and when?

Note: If they say no, this is an opportunity to mention new vaccines, such as pneumococcal vaccine, rotavirus vaccine, are available


107

Impact Assessment Central/Regional Questionnaire

CENT 70. Are you conducting, or do you plan to conduct, a vaccine impact assessment, i.e. a study to determine if the HPV vaccine is reducing disease burden?

Yes No Don’t know

If yes, give details __________________________

If no, why not? __________________________

General Impressions Central/Regional/District Questionnaire

GEN 71. How well was the HPV vaccine accepted? If there were any problems, please comment for each group.

Note: Was it considered to be a safe and effective, and needed vaccine?

New vaccine well accepted

Health-care workers Y N

Professional societies Y N

Community/public Y N

Government Y N

Media Y N

On what is your answer based? _________________

Discuss any problems _________________________

GEN 72. Were there financial implications in introducing the HPV vaccine for each of the following areas?

Ask about the financial implications of each of the following:

Cold chain Y N If yes, specify:____________________

Vaccine transport Y N If yes, specify:______________

Wastage Y N If yes, specify:________________

Communication materials/media Y N If yes, specify:_______

Training Y N If yes, specify:____________________

Other costs? Y N If yes, specify:____________________


108

GEN 73. What effect has the introduction of the HPV vaccine had on your EPI programme?

Please check one that best describes the introduction:

Improved the EPI programme. Please explain ________________________________________

Made the EPI programme worse. Please explain ________________________________________

No effect. Please explain ________________________________________

GEN 74. In your opinion, was the introduction of the HPV vaccine a smooth process or problematic? Please explain.


Very smooth. No problems

Smooth, minor problems. Please explain ________________________________________

Somewhat smooth, some major problems. Please explain _________________________________

Not smooth at all, some major problems. Please explain _________________________________

GEN 75. Many other countries will be introducing this and other new vaccines soon. What have you learned from this experience, and what advice do you have for other countries to ensure a smooth introduction?

Observation of Vaccine Storage Area at the Central/Regional/District Levels

Central/Regional/District Questionnaire

CENT 76. Are all freezers and refrigerators clean and functioning properly?

Y N

CENT 77. Are there thermometers outside the freezers and refrigerators?

Y N Some

CENT 78. Are there thermometers inside the freezers and refrigerators?

Y N Some

CENT 79. Is the temperature inside the refrigerators currently between +2° and +8° C?

Y N Some


109

CENT 80. Is there a log of freezer and refrigerator temperatures?

Y N Some

If yes, has temperature consistently been between +2° and +8°C for refrigerators in the last two months?

Y N Some

CENT 81. How often are temperatures recorded?

Twice daily Daily

No records Other (specify) ____________________

CENT 82. Are temperatures monitored and recorded on weekends and holidays?

Note: Check specifically for holidays in ____________(insert date of most recent holiday).

Y N Sometimes

CENT 83. Are all vaccines arranged as “First expiry, First out”?

Y

N If no, why not? _______________________

Not applicable. Why? ____________________

CENT 84. Did you observe any expired vaccines?

Y N

If yes, which vaccine, and how many?

_______________________________

CENT For vaccines with a VVM

85. Did the VVMs that you observed indicate that vaccine is usable, i.e. Stage 1 or 2

Yes, all vaccines usable

No, some vaccines Stage 3 or 4 (unusable)

Specify vaccine and proportion unusable

_______________________________

CENT For vaccines with a VVM

86. Are vaccines with VVM in Stage 2 arranged so that they are used first?

Y N Not applicable, no Stage 2

CENT 87. Are there spaces between the vaccine boxes/trays to allow air circulation?

Y N


110

CENT 88. Is injection equipment stored in good condition?

Adequate space Y N

Clean and dry conditions Y N

Well organized (i.e. easily accessible) Y N

Other observation (specify) ___________________________________

Notes and Comments

GEN If you were unable to visit the cold store or dry store area, please mention reason.

Record any interesting positive or negative anecdotes or comments by immunization staff.


111

Questionnaire 1.2 — health facility

OBLIGATORY immunization ONLY

Date of interview: ________________ Name of interviewer: _____________________

This questionnaire was conducted at

Region: ______________________

District: ______________________

Health-facility name: ___________

Type of health facility (check one):

Health Centre/Clinic Health Post/Outpost Other (specify) ____________________

Name(s) and title(s) of person(s) interviewed (please list all the persons that you interviewed): EPI Senior Nurse/Health-care worker responsible for vaccinations (or their deputy) should be interviewed

Name________________________ Title _______________________

Name________________________ Title _______________________

Name________________________ Title _______________________

Contact details of most senior person:

Telephone __________________ e-mail address _______________________

Denotes Suggested Key Finding (see Appendix 3).


112

Documents to request at beginning of interview: (check appropriate boxes)

Document / data Document received

Document reported to exist but not

available at time of interview

Document unavailable

Introduction plan for new vaccine

Training materials/reference documents utilized during new vaccine training

Vaccine management guidelines

Media campaign/social mobilization/education materials (brochures, posters, pamphlets, etc.)

Vaccine stock records

Supervisor's book/site visit reports

Injection safety/waste-management policy document

Wastage reports

AEFI protocol/reporting form

AEFI logbook/registry

Sample health card/immunization card

Immunization logbooks, monitoring forms, tally sheets, vaccine registries

Abbreviation Pre- Implementation Planning Health-Facility Questionnaire

GEN 1. Were you (interviewee) working at this health facility at the time of the HPV vaccine introduction?

Yes No

Interviewer: If "No", try to get a staff member who was present when the new vaccine was introduced to participate. If not, continue with the interview although it may not be possible to answer all questions.

GEN 2. When was the HPV vaccine first administered at this health facility?

(MM/YYYY) _________/______

Don’t know


113

Training Health-Facility Questionnaire

GEN 3. Please describe health-facility staff training for the HPV vaccine introduction, if any.

How many people from this health facility were trained? ______________

Who from this health facility was trained? _________________

How many of them are still working at this health facility? _________

How long was the training for health facility staff? __________

What were the key topics covered in the training? _____________

Were there any opportunities to practice the new skills to administer the new vaccine correctly? _____________________

Did the person from this health facility who was trained, train others in the health facility? Yes No Don’t know

Was training conducted before vaccine introduction Yes No

If yes, how long before? _________________

Was training conducted after vaccine introduction Yes No

If yes, how long after? ____________________

Who conducted the training for health-facility staff? __________________________

Other comments on training _________________________

GEN 4. Do you think there are any ways in which the training could be improved for next time?

Yes No. Don’t know

If yes, please describe ___________________________

GEN 5. Are HPV vaccine introduction guidelines or educational and reference materials from the training available?

Ask to see samples.

Yes No Don’t know

Key Finding: Guidelines/training materials provided?


114

GEN 6. Overall, were you satisfied with the training provided?

Yes No Not applicable

Key Finding: Satisfaction with training?

Vaccine Delivery Central/Regional/District Questionnaire

GEN 7. Is informed consent required? Yes No

GEN 8. If yes, does it pose any issues? Yes No

GEN 9. What methods were used for recall or to enhance completion of 3–dose schedule?

Please list the methods

Vaccine Coverage Health-Facility Questionnaire

GEN 10. What is the size of the target population for HPV immunizations in this health facility?

What is the source of this figure?

Girls 12 years of age:______________________

Source of data _______________________

GEN 11. What formula do you use to calculate vaccine coverage? Include the source of the numerator (doses administered) and denominator (target population).

Formula

Numerator source _____________________

Denominator source___________________

GEN 12. What was RUBELLAvaccine coverage among girls 14 yars of age in the year before the HPV vaccine introduction?

Note: use 2009 data.

Rubella _________ year _________

GEN 13. What is the coverage of the first, second and third dose of the HPV vaccine for the most recent administrative period?

HPV vaccine first dose (HPV-1) coverage ______________

HPV vaccine second dose coverage ___________

HPV vaccine third dose (HPV-3) coverage _______________

GEN 14. How often do you report immunization data to the higher level? Ask to see a report.


115

GEN 15. Have immunization registries/child health cards, etc. been updated to include the HPV vaccine?

Check box if updated

Vaccine registry/logbook

Health card

Tally sheets/district reporting forms

Vaccine stock forms

Other (specify)

GEN 16. How many days a week does your site perform outreach immunization sessions, i.e. immunization sessions not conducted at the health facility?

Outreach not performed

_______ times per week

GEN 17. How outreach data are collected?

GEN 18. What changes, if any, did you have to make to outreach sessions when you introduced the HPV vaccine?

No changes required

More vaccine carriers required

Increased number of outreach sessions

Other changes (specify)

Cold-Chain Management Health-Facility Questionnaire

GEN

19. How vaccines are stored at your health facility?


Cold storage box

Refrigerator

Other (specify) _________________

20. What cold chain equipment is utilized during out reach services?

GEN 21. The last time there was an interruption in your power supply, what did you do?

GEN 22. Discuss any changes you had to make in the cold chain before introduction of the HPV vaccine.

Note: Try to distinguish cold-chain expansion/replacement of equipment that is part of normal cold-chain rehabilitation from changes specifically for the new vaccine.


116

GEN 23. Were there any problems with the cold chain recognized after the introduction of the HPV vaccine? If yes, what were the problems and have the problems been addressed? If they have been addressed, how were they addressed?

No problems

Inadequate space

Frozen vaccine

Malfunctioning refrigerators

Power supply

Other (specify)

How resolved?

_______________________________________

Key Finding: Percentage health facilities observed or reported problems with the cold chain

Vaccine Management, Transport & Logistics

Health-Facility Questionnaire

GEN 24. Do you have immunization policy guidelines for vaccine management? If yes, have they been updated to include the HPV vaccine? Please provide a copy at time of interview.

Yes No

GEN 25. How do you forecast vaccine requirements?

GEN 26. How did estimated requirements change following introduction of the HPV vaccine?

Yes No Don’t know

If yes, why? ___________________

GEN 27. Please describe how vaccines are ordered and delivered to the health facility.

Who orders? _______________________________

How often are vaccines delivered? _____________

Any problems with this? ______________________

GEN 28. Have you had any vaccine expirations in the last six months? If yes, what did you do with the expired stock?

Yes No

If yes, action taken_________________________________


117

GEN 29. Have you had any vaccine with VVM in Stage III or IV in the last six months? If yes, what did you do with these vaccines?

Yes No

If yes, action taken _________________________________

GEN 30. Did you run out of any vaccines, including the HPV vaccine, or vaccines supplies in the past six months?

Yes No

Yes, vaccines (specify)

Yes, vaccine supplies (specify)

No

If yes, how many weeks? ___________

If yes, reason for stock out _____________

Key Finding: Percentage of health facilities reporting vaccine or supply stock out in last six months

GEN 31. Are vaccine orders/deliveries tied to injection supplies (i.e. bundling)?

Note: Look at stock records to get this information.

Yes No

Verified by checking stock records

Yes No

Waste Management and Injection Safety


GEN 32. Did you have to make any changes to your waste-disposal system for introduction of the HPV vaccine? If yes, explain.

Yes No

If yes, explain ______________________________

GEN 33. Have you experienced any problems with your waste-disposal system?

Observe site.

Yes No

If yes, explain ______________________________


118

Vaccine Wastage Health-Facility Questionnaire

GEN 34. What formula is used to calculate vaccine wastage and what is the source of the data.

Ask for wastage report.

Vaccine wastage not calculated

Formula:

(See Table 10)

Data source, numerator ______________________

Data source, denominator ____________________

Is formula provided correct? (See Table 10)

Yes No

Source of data:

Stock books

Summary sheets

Other

Key Finding: Wastage report on site?

Yes No

GEN 35. What is the vaccine wastage rate of the HPV vaccine?

Note: If vaccine wastage rate is unknown for new vaccine because PIE is done before administrative data are available, record anecdotal reports or attempt part-year calculation.

HPV vaccine wastage (this administrative period) _____________%

GEN 36. Did you change anything about the way you administer vaccines, to reduce wastage of the HPV vaccine?

Monitoring and Supervision Health-Facility Questionnaire

GEN 37. How many times in the past six months have you received a supervisory visit from district or regional level or from a partner agency? Was the visit documented?

Ask to see the supervisory book, copy of last report.

Number of visits ___________

Is there a written report of the visit? Yes No

Key Finding: At least one documented visit

Yes No


119

GEN 38. If yes, who visited, and what were the problems identified?

Who visited? _______________________________ (job title)

Problems identified

_________________________________

Adverse Events Following Immunization (AEFI)


GEN 39. Do you have a system and written protocol for monitoring and reporting AEFIs for all vaccines? Please describe the procedure.

Ask for a copy of the AEFI protocol and reporting form.

Yes No

If no, why not? ______________________________

Key Finding: AEFI system/protocol in place?

GEN 40. Did you make any changes to the AEFI protocol specifically for the HPV vaccine?

GEN 41. Have you had any reported AEFIs for the HPV vaccine or another vaccine since the HPV vaccine was introduced?

Note: Verify using AEFI log book/registry, if one.

Yes No Don’t know

If yes:

How many for the HPV vaccine? _______________

How many for a traditional vaccine? (specify) _____________

What were the AEFIs? ______________

How were they handled? __________________

Advocacy, Communication & Acceptance


GEN 42. Did you have an official launch ceremony at this health facility at the time of the HPV vaccine introduction?

Note: What did it involve, was it successful, did it get much media coverage?

Yes No Don’t know

If yes, describe ________________________


120

GEN 43. Did this health facility provide any health education messages or materials to the community about the HPV vaccine at the time of introduction?

Ask to see copies of materials.


None provided

Posters

Brochures

Health education sessions

Public meetings

Other (specify)

GEN 44. Did you experience any resistance from the community regarding the HPV vaccine?

Yes No Don’t know

GEN 45. Do you remember any media focus (e.g. on radio, television or newspapers) on the HPV vaccine?

Yes No

If yes, describe ______________________________

Health-Care Worker Knowledge (ask HCW, not head of health facility)


GEN 46. What is the immunization schedule for the HPV vaccine?

GEN 47. What disease(s) does the HPV vaccine prevent?

Interviewer:

HPV cervical precancer and cancer caused by serotypes included into vaccine (around 70% of all). For quadrivalent vaccine – anogenital warts.

Interviewer: Write exact response given

Key Finding: Percentage HCWs that knew what disease(s) the new vaccine prevents?


121

GEN 48. What information do you provide to girls before and after vaccination with the HPV vaccine?

Check if mentioned — don’t prompt but can tell afterwards

Name of the vaccine

Diseases it protects against

Benefits to the girl

Vaccine schedule/when to return

Normal side effects?

Risk of synkopy (fainting)

What side effects they should return for

Bring vaccination card

Other health messages (specify)

Two or more mentioned? Yes No

Key Finding: Percentage HCWs providing two or more accurate pieces of information to parents? Yes No

General Impressions Health-Facility Questionnaire

GEN 49. Were there any financial implications for the health facility involved in introduction of the HPV vaccine?

Ask about the financial implications of each of the following:

Don’t know

Cold chain Y N If yes,

specify____________________

Vaccine transport Y N If yes,

specify______________

Wastage Y N If yes,

specify________________

Communication materials/media Y N If yes,

specify_______

Training Y N If yes,

specify____________________

Other costs? Y N If yes,

specify____________________


122

GEN 50. What effect has the introduction of the HPV vaccine had on your EPI programme?


Improved the EPI programme. Please explain______________

Made the EPI programme worse. Please explain______________

No effect. Please explain______________

Key Finding: Percentage sites reporting that new vaccine improved the EPI programme?

GEN 51. In your opinion, was the introduction of the HPV vaccine a smooth process or problematic? Please explain.


Very smooth. No problems

Generally smooth, minor problems. Please explain______________

Somewhat smooth, some major problems. Please explain _____

Not smooth. Major problems. Please explain ______

Key Finding: Percentage sites reporting a smooth or very smooth introduction

GEN 52. Many other countries will be introducing this and other new vaccines soon. What have you learned from this experience and what advice do you have for other health facilities to ensure a smooth introduction?

Observations at Vaccination Session Health-Facility Questionnaire

GEN 53. Are (all) vaccines reconstituted correctly (e.g. measles, BCG)?

Y N Unknown

(N = unsafe practice)

GEN 54. Are vaccines stored/handled properly during the session, e.g. clean, organized, vaccine vials outside carrier are in foam pad?

Y N Unknown



123

GEN 55. Are appropriate administration techniques observed (for Gardasil intramuscular injection in the deltoid region of upper arm or the higher anterolateral area of the thigh)

Y N Not observed


GEN 56. Are AD syringes used? Y N


GEN 57. Are needles recapped (look in safety box for capped needles)?

Y N Unknown

(Y = unsafe practice)

GEN 58. Are AD syringes disposed of in a safety box?

Y N Unknown


GEN 59. Is the policy on use of the open multi-dose vial observed?

Date opened marked on vial Y N

Open vial discarded at end of immunization session Y N

Other observation (specify)

___________________

Unknown


GEN 60. Summary: How many unsafe practices, based on questions above, were observed?

Number of unsafe practices _________

Key Finding: Percentage of sites with two or more unsafe practices observed

GEN 61. Are girls observed after vaccination approximately 15 minutes?

Y N Unknown

Observation of Vaccine Storage Area Health-Facility Questionnaire

GEN 62. Are all refrigerators clean and properly functioning?

Y N

GEN 63. Is there a thermometer outside the refrigerator?

Y N

GEN 64. Is there a thermometer inside the refrigerator?

Y N

GEN 65. Is the temperature inside the refrigerator currently between +2° and +8° C?

Y N What is the temperature? ________


124

GEN 66. Is there a log of refrigerator temperatures?

Y N

GEN 67. How often are temperatures recorded?

Twice daily Daily

No records Other (specify)

GEN 68. Are temperatures monitored and recorded on weekends and holidays?

Note: Check specifically for holidays in ____________ (insert date of most recent holiday).

Y N Sometimes

GEN 69. Are vaccines arranged as “First expiry, First out”?

Y

N If no, why not? ____________________

Not applicable. Why? ____________________

GEN 70. Did you observe any expired vaccines?

Y N

If yes, which vaccine and how many?

__________________

GEN For vaccines with a VVM

71. Did the VVMs that you observed indicate that vaccine is usable, i.e. Stage 1 or 2

Yes, all vaccines usable

No, some vaccines Stage 3 or 4 (unusable)

Specify vaccine and proportion unusable

___________________

Key Finding: Percentage of health facilities reporting with any VVM in Stage 3 or 4.

GEN For vaccines with a VVM

72. Are vaccines with VVM in Stage 2 arranged so that they are used first?

Y N Not applicable, no Stage 2

GEN 73. Are there spaces between the vaccine boxes/trays to allow air circulation?

Y N

Health Communication Health-Facility Questionnaire

GEN 74. Are any posters or other literature about the new vaccine noted in the health facility?

Y N


125

Stock Room Health-Facility Questionnaire

GEN 75. Is injection equipment stored in good condition

Adequate space Y N

Clean and dry conditions Y N

Well organized (i.e. easily accessible) Y N

Other observation (specify) _________________

Waste Disposal Health-Facility Questionnaire

GEN 76. How are used AD syringes being disposed of?

(If not observed, ask how boxes are disposed).

Safety box

Open bucket

Other

Other observations

GEN 77. How are used safety boxes disposed of?

(If not observed, ask how boxes are disposed).

Note: Specify whether box is emptied and reused or destroyed with contents inside.

Incinerator

Pit-burned

Pit-exposed

Pit-buried

Above-ground area

Box reused

Other observation

GEN 78. Were discarded needles and syringes observed on the ground outside the facility?

Y N

GEN 79. Is waste-disposal site closed off? Y N

Key Finding: Percentage of health facilities with clean, closed-off disposal sites

GEN 80. Describe any other observation of the disposal site.

Notes and Comments

If you were unable to visit the cold store or dry store area, please mention reason.

Record any interesting positive or negative anecdotes or comments by health-care workers.


126

Questionnaire 1.3: — Interview with girls

Optional

Date of interview: ______________________ Name of interviewer: _________________

Region: _________________District: _____________________Health-facility name school name where the girl was interviewed : ______________________________________

Interview girls who have just received the HPV vaccine (can also talk to a group of girls waiting to be vaccinated to get their impressions). Begin the interview by saying the following “I would like to ask you a few questions about the vaccines you received today. The answers you give will help us learn more about how to introduce a new vaccine.” (N.B. You may need someone conversant in the local language to ask the questions).

1. Do you have your Health card with you today? If yes: May I please see it?

Use health card to answer the following

Health card present Yes No

Vaccines received today

HPV

I the health card updated?

Old health card (not updated to include new vaccine)

Old health card (with new vaccine written in by hand)

New health card (updated to include new vaccine)

2. What vaccine(s) did you receive today? Check one box

Names all vaccines (answer correct)

Names some vaccines (partially correct)

Does not know

Mentions specific health benefit of vaccine (e.g. for HPV vaccine says, “got vaccine to prevent cervical cancer”)

Mentions general beneficial effects of vaccines, e.g. ”I got vaccines to be healthy”

Other (specify) _____________________


127

3. Do you know about the HPV vaccine?

Yes No

If yes, which disease(s) do they prevent?

Does not know

Answer correct

Answer incorrect

4. If yes to question 3. How did you receive the message about the HPV vaccine?

Note: Radio, newspaper, television, health-care worker, friend, public meeting.

5. Do you know when to come for your next vaccination?

Note: If answer is no or yes but incorrect, please advise girl when next vaccination is due.

Yes (answer correct)

Yes (answer incorrect)

No

6. Do you know what reaction you may get following your vaccination today?

Note: This question is trying to differentiate between baseline knowledge and knowledge received at current vaccination session.

Yes (answer correct)

Yes (answer incorrect)

No

Interviewer: If answer is no or yes but incorrect, please advise mother of potential injection side adverse experiences, e.g. mild redness, pain, mild swelling at injection site, mild fever.

7. Other comments or observations. Record any interesting positive or negative anecdotes or comments by girls.

WHO assists Member states of the South-East Asia Region to periodically review their surveillance systems and national immunization programmes. These reviews provide an insight into the programme strengths and limitations. Additionally, WHO encourages countries to identify strategies to harness strengths and utilize the available resources to improve the quality of surveillance and immunization. In March 2011, national and international experts reviewed the Expanded Programme on Immunization (EPI) and conducted a post introduction evaluation for HPV vaccine in Bhutan.

This report summarizes the progress made in vaccine preventable disease surveillance, immunization service delivery and coverage, injection safety, vaccine supply, cold chain management, and advocacy and communications. It also provides recommendations for the consideration of the Government of Bhutan and development partners in their efforts to achieve the national goals for immunization.


World Health House Indraprastha Estate, Mahatma Gandhi Marg, New Delhi-110002, India