Development and Implementation of a National Multisectoral Output Monitoring System (SHAPMoS) for HIV Responses in Swaziland:  Challenges and lessons

learned

By Mduduzi Patrick Dlamini (SHAPMoS Manager) & Mduduzi Ndlovu (M&E Officer)

Monitoring & Evaluation UnitNational Emergency Response Council on HIV/AIDS

(NERCHA), Swaziland.

Presentation Outline

• Context• Introduction to National Multisectoral

M&E System in Swaziland• What is SHAPMoS?• How was it developed & implemented?• What are the major challenges

(development & implementation)• Lessons learnt

context• “Three ones” concept to manage HIV response at

country level– one national HIV strategic and action plan,– one national AIDS coordinating authority,– one national HIV monitoring and evaluation system

• National Emergency Response Council on HIV and AIDS (NERCHA) established in 2003 to coordinate HIV response

• National Strategic Plan covered the period 2000 to

2005. New NSP launched in July 2006. • National Multisectoral HIV monitoring and evaluation

(M&E) system developed & launched in October 2005

Introduction to National Multisectoral M&E System in Swaziland

• Goal– Track spread of epidemic– Track effectiveness & efficiency of response– Track inputs provided

• Linked to National Strategic Plan – (3rd one measures extent of 1st of “three ones”

are being achieved)• National indicators (88 indicators)

– Input ,output , impact & outcome

DATA SOURCES• SURVEYS (impact and outcome level data)

1. Biological HIV surveillance2. Population-based Surveillance3. Quality of HIV services Survey4. Workplace survey5. Young People Policy Index questionnaire6. Condom Availability and Quality Survey7. Vulnerability Survey8. School data from existing surveys

• ROUTINE DATA (input and output level data)

9.9. SHAPMoSSHAPMoS10. Ministry of Health HIV M&E system data

What is SHAPMoS?

• Swaziland’s HIV and AIDS Program Monitoring System – is a national routine data collection system that collects

data about non-medical HIV and AIDS services from all implementing partners (at community level and disaggregated by regions) .

• provides data about outputs (immediate, short-term results) of HIV services delivered in communities (e.g. number of persons trained, number of peer educators,

number of OVC supported, etc.)

Developed & implementation of SHAPMoS1. National M&E Capacity Assessment (2004)was conducted.

– lack of M&E capacity in the country in general.– Few practitioners.– only 52% of NGOs had received M&E related training.

2. consultative meetings with HIV implementing partners. 3. Establishment of M&E Technical Working Group (July 2003)

composed of representatives from all stakeholders. Mandate:

– M&E Framework• indicator development Operational level• Review progress on implementation of M&E work plan• Provide input in content of information products• Guidance on training needs • Progress on SHAPMoS implementation• Guide research• Drive development of UNGASS Report

Developed & implementation of SHAPMoS

5. Development of supporting documents for training & reference materials

• Training plan and curriculum

• SHAPMoS guidelines What should be collected & How?

• Operational Plan

• Data auditing & supervision plan

6. Design & piloting of SHAPMoS Forms – Printing of SHAPBooks

– Triplicate (white, blue, green copy)

– Pages /forms to cover 2years

SHAPM o S F o cal PERSO N(HIV IM PL EM ENT ERS)

NERCHA M &E Un it

S end the S HA P M oS form toHIV Coordinating B ody and to

ReM A C

Conduc t data audits to 20% oforganisations that subm itted

S HA P M oS Form s

Record the nam es of H IVim plem enters that have subm itted

S HA P M oS Form s

P repare Q S CR

Reg io n al H IV/A IDS M &ECo o rd in ato r (ReM AC)

A ppoint / nom inate theS HA P M oS Focal P erson

HIV CO O RDINAT ING BO DY

Capture S HA P M oS data onS HA P Data

Verify com pleteness ofS HA P M oS Form s

Capture results of S HA P M oS dataaudits on S HA P Data

S end Q S CR back to regionsfor dissem ination

A ttend quarterly H IV and A IDSfeedback workshops

Record the nam es of H IVim plem enters that subm itted

S HA P M oS Form s

Record S HA P M oS data

A ss is t ReM A C w ith organis ingquarterly H IV/A IDS workshops

Identify organisations that havenot reported on S HA P M oS or

that are s truggling w ithS HA P M oS reporting S end S HA P Data to NE RCHA

M & E Unit

D issem inate Q S C Report

O rganise quarterly H IV and A IDSfeedback workshop in the region

Conduc t H IV and A IDS feedbackworkshop in the region

S end S HA P M oS data audit reportto NE RCHA M & E Unit

P repare S HA P M oS data auditreport

P repare S HA P M oS Form

A ttend quarterly H IV and A IDSfeedback workshops

Capture indicator scores inCRIS database

S end an elec tronic copy ofindicator scores to CS O

Rev iew data audit results

P rov ide nam es of theseorganisations to Regional

H IV/A IDS M & E Coordinators

Conduc t partic ipatory superv is ionv is its

Implementation of SHAPMoS

• Pool of 15 Training of trainers trained (February 2006). – 10 trainers contracted .

• Training list developed with assistance from sectors and 2004 directory.

• A series of training conducted– 3-day training

• 1-day follow-up mentorship sessions– Formally introduce SHAPMoS– Appointment of focal person– Development of reporting tools

• Advocacy sessions– Workshops for head of implementing partners & agency– Other forums

Implementation of SHAPMoS (cont.)

• design & print communication materials – Posters, brochures

• Reporting – First reporting (April – June 07), rate of 35 %– Second reporting (July – September 07) , rate of 60 %– Quarterly Service Coverage Report (printing stage)

Challenges during the development of SHAPMoS

A. Different approaches to M&E by agencies and practitioners

B. Indicator development

C. Data element definitions (piloting in workshop)

D. Other NERCHA departments involvement

E. Lack of M&E Framework for Ministry of Health• M&E unit formed in October 2005

Challenges during the implementation of SHAPMoS 1. Lack of funding and awareness of M&E

– Low awareness of importance of M&E– No budget (7-10% of budget devoted to M&E).

2. Lack of M&E Capacity & staff (2004 assessment by

CADRE) – Nation-wide trainings & mentorship– Reporting requirements of donor agencies– No M&E system in organizations – sending wrong person for training

3. Unrealistic expectations– Funding opportunities– Employment of an M&E officer

4. Lack of enthusiasm to report – National Aids Policy & NSP– Difficulty in enforcing requirement– Engagement of sectors to influence culture of reporting

Challenges during the implementation of SHAPMoS

5. Negative Perceptions • Funding agent versus coordination authority

6. Lack of a best practice in M&E • Data use for decision making • …“SHAPMoS will be expensive”…

7. Lack of a list of HIV implementers • Newspaper advertisements• sectors

8. Organizational structure in NERCHA• M&E unit not under programs unit

9. Report writing • Clinical routine data

Lessons relating to the Development of SHAPMoS

1. Development of two separate curricula– Training – Management

2. Importance of stakeholder participation & advocacy• At all stages

3. Decentralization of government services• In-line with government policy• Regional HIV secretariat employed

4. Paper-base approach system– Ensures that even community-based organizations report

Lessons learnt during the implementation of SHAPMoS

• Advocacy – Heads of implementing partners before training commences

• Mentorship– To assist in aligning data collection tools

• Data audits– 10 % of organizations that reported.– Data quality– Ensure that reported results correlate with org. records

• Collaboration with key stakeholders– Data interpretation– Dissemination

Thank you!!!