mid-term evaluation of the "performance enhancement ... - eeas
TRANSCRIPT
AESA Consortium
1
25639
Framework Contract Beneficiaries 2013 Lot 1
Delegation of the European Commission to Zambia
“MID-TERM EVALUATION OF THE "PERFORMANCE ENHANCEMENT PROGRAMME"(10
TH EDF) AND FORMULATION OF THE "ENHANCED
CAPACITY OF THE MINISTRY OF AGRICULTURE FOR BETTER SERVICE DELIVERY TO FARMERS PROJECT" (11
TH EDF)”
Contract N° 2014/345932
MID-TERM EVALUATION REPORT
November 2014
This programme is funded by the European Union Programme implemented by
The contents of this publication is the sole responsibility of AESA Consortium and can in no way be taken to reflect the views
of the European Union
AESA Consortium
AESA Consortium
DELEGATION of the EUROPEAN COMMISSION
ZAMBIA
“Mid-Term Evaluation of the "Performance Enhancement Programme"(10th EDF) and Formulation of the "Enhanced Capacity of
the Ministry of Agriculture for Better Service Delivery to Farmers Project" (11th EDF)”
Contract N° 2014/345932
Framework Contract Beneficiaries 2013 – LOT 1
MID-TERM EVALUATION REPORT
November 2014
Team Composition: TEAM LEADER: PAUL SIJSSENS
EXPERT 2: DR. SUSAN WERE MAKOKHA EXPERT 3: ELIZABETH VISSER
This report was prepared with financial assistance from the Commission of the European Communities. The views expressed are those of the consultant and do not necessarily represent any official view of the Commission or the Government of Zambia.
AESA Consortium
2
TABLE OF CONTENTS
Table of Contents
EXECUTIVE SUMMARY ...................................................................................................... 3
LIST OF ACRONYMS .......................................................................................................... 7
1. INTRODUCTION ....................................................................................................... 9
1.1 The purpose of the mid-term evaluation and its scope ............................................... 9 1.2 Evaluation process .................................................................................................... 9 1.3 Methodology ............................................................................................................ 10
2. CONTEXT ............................................................................................................... 12
2.1 The agriculture and livestock policy context in Zambia ............................................ 12 2.2 Poverty in Zambia ................................................................................................... 13 2.3 EU policy context ..................................................................................................... 14 2.4 Other cooperating partners ...................................................................................... 14
3. FINDINGS ............................................................................................................... 15
3.1 Programme implementation ..................................................................................... 15 3.2 Programme Design: Overall Objective and Project Purpose .................................... 18 3.3 Result Area 1: Change management and service delivery improved ....................... 21 3.4 Result Area 2: Sector policy, planning and financial management improved ........... 27 3.5 Result Area 3: Human resource management and ICT improved ............................ 32 3.6 Result Area 4: Monitoring and evaluation improved ................................................. 42 3.7 Cross cutting issues, communication and visibility ................................................... 43
4. CONCLUSIONS ...................................................................................................... 45
4.1 Relevance ............................................................................................................... 45 4.2 Efficiency ................................................................................................................. 47 4.3 Effectiveness ........................................................................................................... 49 4.4 Impact ..................................................................................................................... 51 4.5 Sustainability ........................................................................................................... 51
5. RECOMMENDATIONS ........................................................................................... 54
5.1 Recommendations for the remaining lifetime of PEP ............................................... 54 5.2 Recommendations for the 11th EDF follow-up programme ...................................... 55
ANNEXES .......................................................................................................................... 56
Annex 1 Terms of Reference ..................................................................................... 57 Annex 2 Itinerary: people and institutions consulted................................................... 58 Annex 3 Documentation consulted ............................................................................. 63 Annex 4 Key evaluation questions ............................................................................. 64 Annex 5 Reference model of Change Management ................................................... 65
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 3 of 65
Executive summary
i. The "Support to Agriculture Sector Performance Enhancement Programme" (PEP) was
formulated in 2009, to address the observed underperformance of the agricultural sector,
as a major driver of economic growth and poverty alleviation, and the institutional
weaknesses of the Ministry of Agriculture and Livestock (MAL).
ii. The programme, with a budget of €8.9M, aims to ensure that "the agricultural sector
contributes to growth and poverty reduction in Zambia" (Overall Objective) by
strengthening the capacity of the now MAL so that the latter is "able to create a
conducive environment and to deliver appropriate services to spur agricultural growth"
(Project Purpose).
iii. The Financing Agreement (FA) for PEP was signed on 1 July 2011 and the programme is
implemented through three programme estimates (PEs) under the 10th
European
Development Fund (EDF) for a period of 54 months, ending on 13 August 2015. The end
date for the entire programme, including the closure phase, is 31 December 2016.
iv. The current document presents the findings of the Mid Term Evaluation (MTE) that was
carried out between Monday 11 August and Wednesday 3 September, with a validation
workshop on Friday 12 September. At the time of the present MTE, the programme had
been operational for three years (with two years of implementation), with only one more
implementation year ahead.
v. With respect to relevance, the MTE is of the opinion that support to the Agricultural
Sector was and is still relevant. Important strategic documents, such as Vision 2030 and
the FNDP, the 9th
and 10th
EDF CSP, all highlight the importance of this sector. The
SNDP (2011-2015), developed during the implementation period of PEP, features the
theme ‘’Sustained economic growth and reduced poverty” and agriculture has been
identified as the main priority sector in achieving both economic growth and reduced
poverty.
vi. The Financing Agreement (FA) and later documents present the following four results:
1. Change Management and Service Delivery Improved
2. Sector Policy, Planning and Financial Management Improved
3. Human Resources Management and ICT Improved
4. Monitoring and Evaluation Improved
vii. The MTE found that the design of the programme was flawed since the start and that this
has not been sufficiently remedied. Each result is composed of two or more separate
results and results contain activities that could be considered results. This non-prioritised
list of results demonstrates the absence of a logical sequence. This had many negative
repercussions on programme implementation.
viii. In absence of a clear implementation strategy and path, the programme was bound to be
activity oriented, without sufficient consideration for the coherence between the different
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 4 of 65
activities. Combined with the absence of suitable indicators and of baseline data, it is
almost impossible to assess effectiveness and impact of the programme.
ix. According to the FA, a Change Management Team (CMT) was to be set up to implement
PEP on a day-to-day basis. In practice, the CMT appears to be more of a coordination
unit at the headquarters rather than the change champion with sustainable mechanisms to
institutionalise the change across the various levels of MAL through the PACOs,
DACOs, down to block and camp level. Furthermore, much of the day-to-day
management is transferred to the Technical Assistance team.
x. With respect to result area 1: Change management and service delivery improved, it can
be observed that no implementation strategy is available to clarify why these two rather
separate topics are combined in one result. Furthermore, the characteristics or indicators
of the envisaged change have not been articulated. Improved service delivery features
also in the PP.
xi. Activities in result area 1 include the establishment of Change Management Mechanisms,
Strategic Leadership and Management Development and a comprehensive Services
Analysis exercise. To date, a CMT exists at national level; provincial CMTs (PCMTs)
were established in each province and have met once, but have not been actively involved
in implementation. Strategic leadership training was provided for senior level staff.
Anecdotal records are positive, but no monitoring tools to assess changes in knowledge
attitude or practices have been applied. Similarly, the effectiveness of the provided mid-
level management and supervisory skills training cannot be ascertained.
xii. The Services Analysis exercise (focused on districts and stakeholders of the agricultural
sector), as an input for a Functional Review of MAL, was done by MAL as a pre-cursor
to the development of the Strategic Plan, but could not yet be implemented due to the fact
that the Strategic Plan 2014-2016 had not yet been formally endorsed.
xiii. Activities under result area 1 have been expanded to include support to the National
Extension Service Strategy Forum, the Strengthening of Regional Research Stations (to
perform Soil, Seed, Fertiliser and Feed Analysis), the Design and Sourcing of Resource
Packs for Camp Extension Officers and support to the JICA funded RESCAP programme
in order to support harmonisation of extension messages. This demonstrates both the
flexibility of the programme as well as the lack of a proper sense of direction.
xiv. Result Area 2: “Sector policy, planning and financial management improved” includes
topics such as Strengthening Sector Dialogue, Strengthening Policy Analysis Capacity,
Carry Out Public Expenditure Tracking Survey and Enhanced Activity Based Medium
Term Expenditure Framework.
xv. Several activities were undertaken, but most not sufficiently followed through. Sector
Dialogue has concentrated on two coordination meetings at national level (Ag-SAG) but
the PACO, DACO and District Agricultural Committees (DAC) meetings have not yet
taken place. Combined with the non-functioning of the PCMTs, this is an indication of
the challenge of PEP to impact at lower levels of administration.
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 5 of 65
xvi. A sector policy study was done but this has not (yet) resulted in actual policy
harmonisation. The facilitation of the sector Strategic Plan (SP) was funded under this
component, but, as mentioned above, the plan awaits formal approval by the Minister.
xvii. PEP has been supporting the establishment of a Farmers Register in more than 20
districts. Effectiveness as a preparatory exercise for implementation of the e-voucher
scheme is not evident, as long at the methodology for the e-voucher has not been decided
upon and the requirements of the system to be used (re. necessary data, software used) is
not known1. Effectiveness in improved targeting of farmers by extension or other service
delivery agents is likely, but is not being captured.
xviii. The Public Expenditure Tracking Survey and Quantitative Service Delivery Survey that
were promoted by the World Bank and included during formulation are not implemented.
However, PEP was instrumental in allowing MAL to develop Planning and Budgeting
Guidelines in order to provide a common framework to guide all levels of the sector to
develop Work Plans and Budgets. Next, Resource Allocation Formulas are being
developed to assure equitable resource allocations to PACOs and DACOs. This process is
ongoing and not yet concluded.
xix. Support was provided to roll-out the Integrated Financial Management Information
System (IFMIS). MAL employees were identified and trained. After some initial
resistance to install and use IFMIS, it has now been fully embedded in MAL HQ and is
operational. Reportedly the system still suffers of poor functionality of the ICT systems at
MAL HQ but PEP will address this as well (see below). Again, rolling out to Provinces
and Districts has not been done (yet).
xx. Result Area 3: “Human resource management and ICT improved” includes a variety of topics,
amongst which the reconciliation of the Staff Payroll and the provision of Outstanding
Payments of retirement packages for MAL retirees. The expected link between the
payment of outstanding personal emoluments and the recruitment of new staff was not
found, indeed these two events are unrelated: MAL staff is recruited or not regardless of
outstanding payments. The payment by the project (and GRZ) only affected staff retired
up to 2012. Outstanding payments to staff not yet retired were not included, nor payment
to staff retiring after 2012. Therefore the activity solved the situation partly for the MAL
(ex-)staff, not for all and not for ever. In future the problem will disappear as from now
there will be no terminal allowances paid.
xxi. Result area 3 includes also the development and implementation of the HR Strategy. To
date this HR strategy does not exist. A personnel record system is being developed as
well as the HR database. The existing Payroll Management and Establishment Control
(PMEC) was to be extended to provide other required human resource information. This
process is also still ongoing.
1 TA Team Leader responded that the refined farmer register can be migrated to any other database that might be
adopted
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 6 of 65
xxii. Other training provided under this component is the induction training, for Camp
Extension Officers (CEO), which was reportedly much appreciated. With respect to
Improve Planning and Management of Agricultural Education and Training Institutions,
PEP support was limited to the provision of a small quantity of equipment, training of FI
and FTC principals & curriculum development.
xxiii. PEP provided support to renovate and upgrade the MAL website, and introduced a
nationwide MAL email system. Internet connectivity has greatly improved and will
proceed further. A large ICT tender was ongoing at the time of the MTE, and will
contribute to enhanced connectivity with field and district staff. It is imperative that the
remaining programme duration is used to ensure that all ICT equipment is sustainably
used to full potential.
xxiv. PEP supported the updating of the MAL's HIV/AIDS Workplace Policy 2011-2015 as
well as printing and distribution. Leadership sensitisation on HIV/AIDS was provided
and trainers trained. Gender mainstreaming guidelines were developed and a check list
was developed to be used as a hand book on gender mainstreaming.
xxv. With respect to M&E, PEP supported the completion of the M&E manual for MAL
(MAL Monitoring and Evaluation System, July 2013). The report was subsequently
edited by Regional Strategic Analysis and Knowledge Support System (ReSAKSS; MAL
Monitoring and Evaluation Manual, April 2013) and is ready for use, including tools and
indicators. It is hoped that MAL will indeed adopt the M&E framework throughout the
entire Ministry.
xxvi. In conclusion, it can be observed that many activities are still being implemented which
may all constitute a first step towards a major improvement (e.g. ICT, Agricultural
Strategy, training courses). It is important that the MAL/PEP demonstrate that the
subsequent actions are following through, so that indeed the MAL operations can become
more efficient and transparent. Examples are the Functional Review, the HR policy and
right-sizing of staff, the implementation of the M&E system. If this will be accomplished
during the implementation period of PEP, these can be considered major achievements.
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 7 of 65
List of acronyms
Ag-SAG Agriculture Sector Advisory Group
CAADP Comprehensive Africa Agriculture Development Programme
CEO Camp Extension Officer
CP Cooperating Partner
CRIS Common External Relations Information System
CMT Change Management Team
DAC Development Assistance Committee
EDF European Development Fund
EU European Union
EUD European Union Delegation
FA Financing Agreement
FAO Food and Agriculture Organisation
FISP Farmer Input Support Programme
FNDP Fifth National Development Plan
FRA Farmers Reserve Agency
GRZ Government of the Republic of Zambia
HIV/AIDS Human Immunodeficiency Virus/ Acquired Immunodeficiency syndrome
HRD Human Resource Development
HRIS Human Resource Information System
HRM Human Resource Management
ICT Information and Communications Technology
IFAD International Fund for Agricultural Development
IFMIS Integrated Financial Management Information System
INSET In-Service Training
JICA Japan International Cooperation Agency
LCMS Living Conditions Monitoring Survey
LFM Logical Framework Matrix
M&E Monitoring and Evaluation
MAL Ministry of Agriculture and Livestock
MTE Mid-Term Evaluation
NAIP National Agricultural Investment Plan
NAP National Agriculture Policy
NAO National Authorising Officer
NEPAD New Partnership for Africa’s Development
NGO Non-Governmental Organisation
NIPA National Institute of Public Administration
OVI Objectively Verifiable Indicators
PCM Project Cycle Management
PE Programme Estimate
PEP Support to Agriculture Sector Performance Enhancement Programme
PMEC Payroll Management and Establishment Control
PPD Policy and Planning Department
PSC Programme Steering Committee
PS Permanent Secretary
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 8 of 65
PSU Programme Support Unit
RESCAP Rural Extension Service Capacity Enhancement Project
SAPP Smallholder Agribusiness Promotion Programme
SIDA Swedish International Development Agency
SMART Specific, Measurable, Achievable, Realistic and Time bound
SNDP Sixth National Development Plan
SP Strategic Plan
SUPE Start-Up Programme Estimate
TA Technical Assistance
TAPS Technical and Administrative Provisions
TL Team Leader
ToR Terms of Reference
ZARI Zambia Agricultural Research Institute
ZCA Zambia College of Agriculture
ZMW Zambia Kwacha (per 1 January 1, 2013)
ZNFU Zambia National Farmers Union
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 9 of 65
1. Introduction
1.1 The purpose of the mid-term evaluation and its scope
The mid-term evaluation (MTE) of the 10th
European Development Fund (EDF) Support to
Agriculture Sector Performance Enhancement Programme (PEP) was the first component of a
combined assignment, together with the formulation of the 11th
EDF Enhanced Capacity of the
Ministry of Agriculture and Livestock (MAL) for Better Service Delivery to Farmers Project.
The global objective of the assignment reflects the link between the two components that made
up the assignment: “to draw policy-relevant, actionable and evidence-based lessons and
recommendations from a mid-term evaluation of the current PEP to support an effective
implementation as well as the attainment of the expected results by the end of the project and to
ensure that this informs the design of the forthcoming "Enhanced Capacity of the Ministry of
Agriculture for Better Service Delivery to Farmers" project”.
The specific objectives of the mid-term evaluation of PEP were to:
assess whether, and by how far, progress has been made by the project towards fulfilling
the outcomes and objectives, as stipulated in the project log frame and in the description
of actions;
re-assess the strategic objectives and implementation mechanisms proposed during the
design of the project, and the extent to which these have been effective in delivering
results;
provide recommendations for any required change/modification to the project scope
(including objectives, management arrangements, financing, etc.) in order to support
effective implementation and the attainment of the expected results by the end of the
project.
The Terms of Reference of the assignment are given in Annex 1.
1.2 Evaluation process
In August 2014, AESA consortium was awarded the contract to undertake the mid-term
evaluation of PEP and the formulation of a follow-up programme. Although the distinction
between the two components of the assignment cannot always be discerned, this report aims to
limit itself to the evaluation part of the assignment.
The assignment started on 11 August 2014 with five days of home-based preparation. The
mission in Zambia commenced on 18 August 2014. The first days in-country were used for
briefings by the Delegation of the European Union (EUD), National Authorising Officer (NAO)
and the Programme Support Unit (PSU), and meetings at MAL. At the end of this inception
period an inception report was produced. During the inception period it was agreed that a single
inception report would be produced for both elements of the assignment. That same first and
following weeks in-country follow-up meetings were held with MAL staff, EUD and other
programme stakeholders.
From 25-27 August 2014 field work was undertaken in Southern Province. Meetings were held
with MAL staff at Provincial, District and Camp levels, and interviews were held with farmers
and other stakeholders, like private sector agents and Zambia National Farmers Union (ZNFU).
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 10 of 65
Preliminary findings, conclusions and recommendation were presented to EUD, MAL and the
PSU during a recapitulation meeting on 2 September 2014. This was followed by a bilateral
meeting with MAL/PSU during which the main findings were thoroughly analysed and
omissions corrected. The revised findings were subsequently presented and discussed at a
validation workshop on 12 September 2014, that was attended by a wide range of stakeholders
(see attendance list in Annex 2) after which the draft mid-term evaluation report was finalised.
Table 1.1 gives a summary of the PEP mid-term evaluation process. Annex 2 provides a detailed
itinerary.
Table 1.1 Summary of the PEP mid-term evaluation process
Date Activity
11-15 August 2014 Home-based desk study
17 August 2014 Mission team arrives in Zambia
18 August 2014 Briefings at EUD, NAO and PSU
19-29 August 2014 Consultation of programme
stakeholders
1-3 September Production of draft mid-term
evaluation report
12 September 2014 Validation workshop
1.3 Methodology
The mid-term evaluation was organised around the responses to evaluation questions which were
systematically covering the Development Assistance Committee (DAC) evaluation criteria of the
Organisation for Economic Co-operation and Development (OECD), namely: relevance,
effectiveness, efficiency, impact and sustainability.
The key questions, given in Annex 4, were used to guide the evaluation process. Evaluation is by
nature a dynamic process and the questions were therefore interpreted in a flexible manner.
Therefore the questions are labelled “key questions”, providing a reference framework for the
evaluators to ensure all necessary information is collected; they were not used as a questionnaire.
Throughout the assignment the following methodologies were used:
Quantitative methods whenever possible, e.g. regarding financial management of the
programme, payment of personal emoluments and support to the farmers register;
Qualitative methods using semi-structured interviews with key informants and other
stakeholders and focus group discussions;
Field observation;
Literature review;
Contents analysis of available data.
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 11 of 65
It may be observed that the mid-term evaluation came at a late stage of programme
implementation. The programme had been running for over three years and has less than one
year left. Because of delays and slow implementation in the first years, EUD decided to have the
MTE well in the second half of the implementation period, so a more meaningful evaluation
could take place. A consequence is that the MTE not only looked at the Start-up Programme
Estimate (SUPE) and PE1, as indicated in the ToR, but also at the implementation so far of PE2.
Another consequence is that there is little time remaining to make adjustments in the remaining
time of PEP. Most of the recommendations will be more useful for the design of the follow-up
programme under the 11th
EDF.
After this first introductory chapter, Chapter 2 describes the sector and policy context in which
PEP is implemented. Chapter 3 presents the findings of the MTE, by result area. This is followed
in Chapter 4 by the conclusions of the MTE, answering the evaluation questions on relevance,
effectiveness, efficiency, impact and sustainability. The final Chapter 5 gives recommendations.
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 12 of 65
2. Context
2.1 The agriculture and livestock policy context in Zambia
PEP was formulated in 2009 in the context of Vision 2030, the Fifth National Development Plan
(FNDP), the National Agriculture Policy (NAP) and the Zambia Comprehensive African
Agricultural Development Programme (CAADP) Compact. These documents all indicate
agriculture as a priority sector and point at the private sector as vital to reduce the sustained high
level poverty in Zambia. The role of the public sector as defined in the documents above, should
be to provide quality services and effectively implement programmes, as well as policy
formulation, enforcement of legislation and maintaining an environment that is conducive to
agriculture productivity, all underpinned by a sound M&E framework for the sector.
However, although one of the challenges mentioned in the NAP is the limited presence of the
private sector, for instance in marketing and extension, it is not clear what precisely is expected
from the private sector and how an environment is created that will encourage private sector
involvement. The NAP still describes a caring government rather than an enabling government
that allows and creates opportunities and the right incentives for ‘desired’ behaviour.
Since the start of PEP implementation, GRZ has developed its Sixth NDP (SNDP) 2013-2016,
and a revised SNDP, which reemphasise the importance of agriculture in Zambia’s strategy to
address (rural) poverty. The NAP is under revision, a MAL Strategic Plan 2013-2016 was
developed (but not yet approved) and the National Agriculture Investment Plan (NAIP) 2014-
2018 was developed under CAADP.
Based on the revised SNDP, the overall strategic focus for agricultural development is increased
production, productivity and value addition for crops, livestock and fisheries as well as
horticultural products with the aim of improving food security and nutrition for all populations of
Zambia and more importantly to contribute to job creation and poverty reduction. This then
became the underpinning factor for PEP in its endeavour to support MAL.
The NAIP gives some problem analysis, even though it is more descriptive rather than analytical
and some main issues are not really followed through. However, this document does provide
quantitative data and sets specific targets and indicators.
The overview of the current agriculture policy context show a collection of individually
interesting documents, but the relationship to each other, or the hierarchical order, is not clear.
The figure below, from MAL, shows the different current policy documents in a single diagram,
but does not clarify the interrelationships of the policies and strategies. It provides a challenge
for an integrated monitoring of objectives and targets of MAL. It certainly justifies PEP activities
aiming for policy harmonisation, planning and M&E.
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 13 of 65
Figure 2.1 Overview of agricultural policies and strategies in Zambia (source: MAL)
2.2 Poverty in Zambia
At the time of formulation of PEP it was observed that despite the overall good performance of
the Zambian economy, the GDP growth had not translated into any significant drop in poverty
(Zambia-EC Country Strategy Paper (CSP) and Indicative Plan (NIP) for the period 2008-2013).
The number of people living in absolute poverty remained high in Zambia. According to the
Living Conditions Monitoring Survey (LCMS) IV of 2004, 68 per cent of the population fell
below the national poverty line. Rural incidence of extreme poverty fell from 71 per cent in 1998
to 65 per cent in 2004. In urban areas, however, the incidence of extreme poverty declined by 5.6
per cent, from 36 per cent to 34 per cent of the total population. That implies that the incomes of
the poor population are on average 64% of the poverty line. Income is very unevenly distributed
in Zambia. According to the LCMS IV, the Gini coefficient for Zambia was 0.55 in 2004, which
is among the highest levels of income disparities in Sub-Saharan Africa.
At the time of evaluation the poverty situation has changed minimally. The national economy
continued to grow (on average 5.7% per year between 2002 and 2012). However, income
Sector Policy Development Level
National Planning Level
Sector Planning Level
Implementation Level
R-SNDP
NAIP
Strategic Plan
This is an overarching Long-term Policy
framework for the agriculture sector
The R-SNDP is the current medium-term national development plan under Vision 2030. The Agriculture Chapter in the R-SNDP is a summary of the NAIP
The NAIP is an investment plan outlining priority investments areas for the sector
The strategic plan is a medium term implementation plan for the Ministry which reflects the core business of the ministry where attention shall be focused and results expected to be seen
NAP-IP
The NAP-IP is an overarching long-term plan to implement the NAP. This implementation plan outlines the roles and functions of various stakeholders in the sector
ATP
The main goal of the Agriculture Transformation Programme (ATP) is to fast track the execution of sector programmes and strategies as identified in the NAP, NAIP and R-SNDP. It is a tool that the Ministry of Finance can use to mobilize extra resources for the agriculture sector
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 14 of 65
inequality has dramatically increased over the past 10 years and is currently one of the worst in
the world. The Gini index was 57.5 in 20102. The gap between urban and rural poverty continues
to widen3. Figures for 2014 indicate 60.5% of the population living below the poverty line, and
rural poverty at 75%. The percentage of people living in extreme poverty is 42% at national level
and 58% in the rural areas. The agricultural sector has been growing at a volatile rate of as much
as 7% since 2009, but has not been able to contribute to reduce the stagnant high rural poverty
rates.
Nearly half of the country’s rural population has a daily caloric intake of less than 1,750 Kcal per
day, while families spend nearly 80 percent of their incomes on food. Only two percent of all
calories consumed by the average Zambian are from pulses, vegetables, and nuts; highlighting
the dire need for dietary diversity. Shortage of nutrients in diets limits growth, weakens
immunity and increases mortality4. Zambia has one of the highest HIV prevalence in the world
(14.3% of persons aged 15-49 years) and nutrition remains a key component in managing this
condition.
2.3 EU policy context
Under the 10th
EDF CSP-NIP interventions were planned under two focal areas: Regional
Integration/Infrastructure and Transport and Human Development/Health, combined with
Macro-economic support. Non-focal interventions would concentrate on agriculture-food
security, governance, and support to non-state actors. PEP was identified and formulated under
the non-focal programme for food security and agriculture diversification, in particular as
contributing to rural agricultural extension initiatives targeted at realizing production and yield
improvements and stabilisation.
The objective of the non-focal sector “food security and agriculture diversification” was “support
for implementation of the cross-sectoral food security strategy and wider agriculture growth, as
major vehicles for rural poverty reduction under the FNDP 2006-2010”.
2.4 Other cooperating partners
Around ten other Cooperating Partners (CPs) are active in the sector. According to the
formulation document of PEP, the programme was “considered by all agriculture CPs as a
golden opportunity to renew the role of the public sector and give agriculture the core role it
should have in poverty reduction”.
Opportunities for synergy were present with IFAD and JICA. IFAD with funds from SIDA and
the Finnish Embassy supported the M&E system within MAL. JICA implemented the Rural
Extension Services Capacity Advancement Programme (RESCAP), with technical assistance in
the ministry and extension activities in selected provinces and districts.
2 World Bank, Gini index 2014 (Gini index measures the extent to which the distribution of income or consumption
expenditure among individuals or households within an economy deviates from a perfectly equal distribution. A
Gini index of 0 represents perfect equality, while an index of 100 implies perfect inequality.) 3 Draft National Indicative Programme, 11
th EDF, for cooperation between the Republic of Zambia and the
European Union, March 2014 4 Zambia National Agriculture Investment Plan (NAIP) 2014-2018, Under the Comprehensive Africa Agriculture
Development Programme (CAADP), Third Draft Final, Ministry of Agriculture and Livestock, March 2013
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 15 of 65
3. Findings
3.1 Programme implementation
Programme duration
The Financing Agreement (FA) for PEP was signed on 1 July 2011. The execution period was 66
months, starting from the date of signature of the FA: 42 months for operational implementation
and 24 months for closure of the programme (see Table 3.1).
Table 3.1 Main PEP implementation dates
Action Starting date End date Duration
Signature of FA 1 July 2011
Operational implementation 1 July 2011 31 December 2014 42 months
Programme closure 1 January 2014 31 December 2016 24 months
Total execution period 66 months
Source: PEP Financing Agreement
The programme is being implemented through three consecutive Programme Estimates (PEs):
one Start-up PE, PE1 and PE2. The FA specifies that any contracts and programme estimates
implementing the FA should be signed within the first three years after signature of the FA (d+3
rule). While preparing PE2, a request for extension of the Period of Execution from 66 to 78
months was submitted and granted (second Rider to the FA, received on 6 June 2014), which
allows PE2 to run for 18 months rather that the 10.5 months that would have been the maximum
duration under the terms of the original Financing Agreement. This means that all agreements
have to be signed prior to 01 July 2015. The end date of the PE2 is 13 August 2015, which
effectively means that end of the implementation phase. The end date for the entire programme,
including the closure phase, was not altered and remains 31 December 2016. At the time of the
present MTE, the programme has thus been operational for two years, with only one more
implementation year ahead. Table 3.2 and Table 3.3 summarise key information on the PEs.
Table 3.2 Starting and expiry dates of PEP Programme Estimates
PE Starting date Expiry date Duration
SUPE 31 July 2012 30 January 2013 6 months
PE1 11 February 2013 10 February 2014 12 months
PE2 14 February 2014 13 August 2015 18 months
Total 36 months
Source: EUD, CRIS, 19 August 2014
The SUPE was prepared with the help of a short-term consultant between October 2011 and
April 2012 to assist MAL to prepare a Start-Up Programme Estimate (SUPE) which was signed
on 31 July 2012.
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 16 of 65
Programme finance
Table 3.3 and 3.4 give an overview of the budgets and utilisation of PEP funds so far.
Table 3.3 Key financial figures of PEP Programme Estimates
PE Funds committed Amount disbursed Balance Percentage
disbursed5 EUR ZMW EUR ZMW EUR ZMW
SUPE 141,073 934,700 123,664 747,760 17,408 186,940 87.7
PE1 2,868,415 20,614,726 1,912,799 13,799,920 955,617 6,814,804 66.7
PE2 3,708,861 27,988,284 1,364,155 10,816,728 2,344,695 17,171,556 36.8
6,718,349 49,537,710 3,400,618 25,364,408 3,317,720 24,173,300
Source: EUD, CRIS, 19 August 2014
Table 3.4 Utilisation of PEP funds
PE Funds committed Amount disbursed Amount spent Percentage
spent6 ZMW ZMW ZMW
SUPE 934,700 747,760 486,517 52.1
PE1 20,614,726 13,799,920 11,459,922 55.6
PE2 27,988,284 10,816,728 3,209,4447 10.5
49,537,710 25,364,408
Source: PEP progress reports
For SUPE an imprest of ZMW 747,760 was released. At the close of SUPE ZMW 486,517 had
been utilised. The underutilisation of 48% of the committed funds were mainly caused by the
delay in procurement of two vehicles (MZW 366,000) and no expenditure on operating costs
(MZW 520,000) as there were no support staff and vehicles yet.
The 44% underutilisation of PE1 funds is mainly caused by a number of activities that did not
take place in PE1 as planned, in particular Support DACO coordination and DDCC Natural
Resources sub-committee meetings (MZW 455,000), Organisational review with MDD (MZW
574,000; pending the signing of the Strategic Plan) and M&E training (MZW 1,076,250; pending
the completion of Training of Trainers through the IFAD SAPP programme).
Programme Steering Committee
According to the FA, a Programme Steering Committee (PSC) was to be set up to oversee and
validate implementation and should meet at least quarterly.
Since the start of PEP, only two PSC meetings have taken place: one on 20 December 2012 and
one on 17 December 2013. The meetings were held to discuss and approve PE1 and PE2
respectively. According to the minutes the views of the PSC are used to review the draft PEs.
5 Due to different exchange rates over time, percentages are different in EUR and MZW. In this table the percentage
is calculated based on EUR. 6 Amount spent on implementation as percentage of committed funds
7 At 31 July 2014, as informed by TA Team Leader. Thirteen months of implementation remain
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 17 of 65
PEP management
According to the FA, a Change Management Team (CMT) was to be set up to implement PEP on
a day-to-day basis. In addition, an imprest administrator and an imprest accounting officer
should be assigned by MAL to the management and implementation of the programme.
At the start of the programme the Director Policy and Planning was indicated as the Programme
Supervisor and Imprest Administrator and the Chief Accountant as the Imprest Accounting
Officer.
Although the CMT was established during SUPE it didn’t take on the day-to-day management of
PEP. CMT meetings do take place on a quarterly basis, but have more the character of sharing of
information, check on progress of PE activities and authorise all forthcoming activities, including
ToRs. The name of the CMT therefore does not reflect its function. The change that the
programme pursues is not clearly defined. Without a common understanding of the aspired
change, it is impossible to manage it. (See also the remarks in Section 4.1 and the model for
change management in Annex 5).
Even to have regular management meetings of a core PEP management team (consisting of PPD,
HRA, accounts and TA) proved impossible due to the heavy schedule of meetings of MAL staff.
In practice the management of the programme has been handed to the PSU, with the TA team
and Director PPD meeting on a weekly basis. Initial Provincial Change Management Teams
(PCMTs) PCMT meetings were held in all province during PE1, so they were established in
practice, although not yet utilised for provincial and district activities.
There have been frequent changes of the MAL Chief Accountant. So far there have been six
different ones since the start of the programme. Since the Chief Accountant is the PEP Imprest
Accounting Officer this has caused challenges in financial management of the programme.
Following an addendum to the FA the Deputy Imprest Accounting Officer has been changed to
the Programme Accountant in the PSU, although all requisitions for project expenditure are
counter-signed by the Chief Accountant.
For administrative support the FA stipulates that the PEs should include resources for a Change
Management Secretariat (CMS). Contracting a programme accountant and a programme
administrator was planned and budgeted in SUPE, but only started from PE1. The CMS is
housed together with the TA team at the offices of PMTC (Zambia) Limited. The TA team and
the CMS together form the Programme Support Unit (PSU).
The FA also stipulates that the PEs would include resources for a legal expertise team and a
long-term Communication Officer. Neither is in place and they are not foreseen within the
remaining lifetime for the programme. When preparing PEs the need for a legal team or a
Communication Officer was not identified and therefore not included.
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 18 of 65
Technical Assistance
The FA further provides for technical assistance under a service contract. Contracting of the TA
started with a call for Expressions of Interest in June 2011 and the contract was awarded in July
2012. The TA team, comprising the Team Leader (Institutional Development/Change
Management Expert) and the Human Resources Management Specialist is in place from 1
September 2012.
The mission observed that the TA team is very dedicated and committed to the progress of the
programme. With the CMT not assuming the day-to-day management, some of these tasks are
currently executed by the TA.
3.2 Programme Design: Overall Objective and Project Purpose
Introduction
Programme design is an important factor that strongly affects implementation and success of a
programme, right from the start. In the case of PEP, this appears to have been particularly the
case. The following chapter elaborates therefore, in a rather detailed manner, on the design of the
programme; from the moment it was conceived until its present suitability.
Conception of the programme
In 2009, a comprehensive study was done to assess the capability of the then Ministry of
Agriculture and Co-operatives (MACO) to respond to the changing environment and increased
expectations of the agricultural sector8. The findings of this study were consolidated at a
comprehensive participatory review workshop. Consensus was reached on the major capacity
development needs for the Ministry, in order to assume the leadership role in promoting
agriculture as the engine of Growth and Poverty Alleviation in Zambia. These needs were
subdivided in five key areas:
Inform and strengthen policy analysis and decision-making processes
Review functions, role and capacities for services provision and co-ordination
Build modern budget, planning and financial systems from national to district levels
Modernise human resources management and development
Modernise knowledge management and technology (ICT)
The workshop subsequently proceeded to draft an initial design for the PEP programme9. The
Purpose of the Programme was formulated as follows: “to renew and develop MACO’s
functions, systems and capacities for improved leadership, partnerships and performance in
sector policy and strategy formulation; intra- and inter-Ministry planning; programme
development, co-ordination and implementation; and services provision and co-ordination to
farmers”.
8 Assessment of Current MACO Capacity, Capacity Development Needs and Challenges, Strategy and Roadmap for
a Performance Enhancement Programme in MACO/GRZ; July 2009; Cardno Agrisystems for EU 9 Participatory Review of the Ministry of Agriculture and Co-operatives (MACO) Performance in assuming its
Leadership role in promoting Agriculture as the engine of Growth and Poverty Alleviation in Zambia; July 2009,
CardnoAgrisystems for EU
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 19 of 65
The workshop proceeded to identify as many as ten main expected results: 1) Change
Management and Leadership Development; 2) Services Analysis and Functional Review;
3)Policy, Strategic and Sector Planning; 4) Public Sector Budgeting, Planning and Financial
Management; 5) Monitoring, Evaluation and Performance Accountability; 6) Improved
Personnel Management; 7) Improved Planning and Management of MACO Staff Development;
8) Improved Planning and Management of Agricultural Education and Training; 9) Enhanced
Use of ICT in Administration and Service Delivery; 10) Enhanced Response to HIV/AIDS.
This suggests that these two major steps in the identification phase of PEP did not succeed in
analysing the current situation to an appropriate level of detail, and did insufficiently allow the
identification of priorities and cause & effect. The capacity assessment study described the
prevailing situation elaborately, but the perceived needs were still defined in a very general
manner. These shortcomings were not remedied during the stakeholder workshop, where also no
priorities nor a implementation path were identified and where results had merely been defined
as broad areas with room for improvement.
Weaknesses in programme design
The FA and TAPs made an effort to describe the Programme Purpose (PP) in a slightly more
precise manner as follows: “The Ministry of Agriculture and the Ministry of Livestock and
Fisheries are able to create a conducive government and deliver appropriate services to spur
growth in agriculture”. Note that this definition does not correspond to the requirements of a
well-defined PP, which should describe the direct benefits for the target group10
.
The Financing Agreement (FA) and later documents present the following four results:
1. Change Management and Service Delivery Improved
2. Sector Policy, Planning and Financial Management Improved
3. Human Resources Management and ICT Improved
4. Monitoring and Evaluation Improved
It is clear that most of the ten broad result areas that had been identified at the Stakeholder
workshop have simply been combined into four results. Furthermore, those results that could not
be fitted within the four FA results have been incorporated as activities. (e.g. Services Analysis
and Functional Review, Improved Planning and Management of Agricultural Education and
Training, Enhanced Response to HIV/AIDS).
Consequently, each result is in fact composed of two or more separate results and contains
activities that may be considered results. This non-prioritised, long list of results, cannot but lead
to difficulties in defining the logical sequence of activities leading to results and results leading
to Programme Purpose. It is therefore not surprising that the LFM is not considered as a useful
management tool by programme management because the logical sequence is lacking. In
absence of a clear implementation strategy and path, the programme is bound to be activity
oriented, without sufficient consideration for the coherence between the different activities. It
10
Project Cycle Management Guidelines, EU, 2004
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 20 of 65
will be demonstrated elsewhere in this report that this has indeed been the case and that this has
had negative repercussions on programme implementation.
Example
Activity 1.3 in the revised PEP Logical Framework (vs October 2013) is “a comprehensive services
analysis exercise, focused on the districts and the stakeholders of the agricultural sector”. It is also
mentioned that this analysis will serve as an input for the Functional Review of MAL. When the MTE
mission inquired about the status of this exercise, it was reported that the review can only be done upon
approval of the Strategic Plan. This sequence of events (Service Analysis→Strategic Plan→ Functional
Review→….?) is not sufficiently demonstrated in the LFM, nor in the PE. The absence of clear
implementation paths will cause confusion and is likely to result in reduced commitment of the various
stakeholders.
Programme or Project?
The distinction between programme and project can based on many different definitions and the
corresponding discussion will go well beyond the scope of the current report. However, it
should be noted that according to the EU Project Cycle Management Guidelines, a programme is
generally broader and more complex in nature than a project. It is therefore recommended in
these Guidelines that for the design of a programme, interlocking Logical Framework Matrices
(LFM) should be used, whereby a separate LFM is used for each result area.
Project Cycle Management
The SUPE started on 1 August 2012 and the supporting TA contract started one month later, in
September 2012. This means that at the time when the programme became operational, three
years had passed since identification in 2009. It would have been opportune to start
implementation with a planning workshop, not only to reconsider whether previously observed
needs were still valid, (especially considering the reportedly rapidly changing environment), but
also to design an annual work plan in a participatory manner, including key progress indicators.
Revision of the LFM
One exercise to revise the LFM was undertaken in 2012. As a result, the definition of the
Programme Purpose was slightly revised in ’The capacity of MAL is strengthened in an effective
and sustainable way and their operations are improved”. This does not appear to constitute a
major improvement from what it was before (“The Ministry of Agriculture and the Ministry of
Livestock and Fisheries are able to create a conducive government and deliver appropriate
services to spur growth in agriculture”). According to the PCM manual, the definition of the PP
should refer to a higher level achievement rather than to the sum of results- as is the case here. It
also did not explain what is meant with an effective and strengthened capacity of MAL or with
improved operations. Furthermore, as argued above, the shortcomings in design are also evident
in the definition of the results and the absence of logical sequencing of activities and results.
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 21 of 65
This has not significantly changed, although an effort was made to provide some SMART11
indicators.
Objectively Verifiable Indicators (OVIs)
Unfortunately, even the OVIs that are mentioned in the revised LFM display shortcomings.
According to the EU PCM Manual, “Indicators at the level of a result level should not be a
summary of what has been stated at activity level”. This means that OVIs at PP level such as
‘’the number of new policies and legislation & plans approved/promulgated” is not appropriate.
It is not even appropriate at result level, where exactly the same indicator is also provided under
Result 2! This illustrates once more the absence of logical sequencing. It was also noted that still
at the time of the MTE, two years after the revision of the LFM, many of the indicators still had
no baseline or targets and were marked as “to be defined”.
3.3 Result Area 1: Change management and service delivery improved
Table 3.5 summarises the main parameters of Result Area 1.
Table 3.5 Indicators and activities of PEP Result Area 1: Change Management and Service
Delivery improved Indicator Sources of verification Baseline Target
Number of critical units missing
within the MAL structure
Ministry’s organogram TBD 0 units missing
Level of satisfaction of MAL staff
with management performance
Opinion survey TBD Score > 6
% of staff with reviewed job
descriptions
Job descriptions / MAL annual
report
0% 10%
Activities 1.1 Establish Change Management Mechanisms, notably by setting up Change
Management Teams in the Ministry, supported by a long-term adviser and a support unit.
1.2 Promote Strategic Leadership and Management Development, through organisational
development training courses and orientation workshops, study tours and twinning
arrangements in the region.
1.3 Carry out a comprehensive Services Analysis exercise (focused on districts and
stakeholders of the agricultural sector), as an input for a Functional Review within MAL
aimed at renewing roles and streamlining Ministries systems toward improved
organisational efficiency and effectiveness.
Source: Revised PEP Logical Framework (October 2013)
Result Area 1 has been defined as ‘’Change Management and Service Delivery Improved”, but
no implementation strategy is available to clarify why these two rather separate topics are
combined in one result nor how the change management will cause concurrent improvements in
service delivery.
Activity 1.1: Establish Change Management Mechanisms, notably by setting up Change
Management Teams in the Ministry, supported by a long-term adviser and a support unit
11
SMART Indicators: Specific to the objective it is supposed to measure; Measurable (either quantitatively or
qualitatively), Available at acceptable cost, Relevant to the information need of managers, Time-bound
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 22 of 65
Under this activity management components of the programme are set up and supported. These
are the PSC, CMT, PCMT and Change Management Secretariat (see also Section 3.1 above).
Project Steering Committee
A Programme Steering Committee (PSC) has been set up. The PSC is chaired by the Permanent
Secretary (PS) of MAL and is made up of representatives of MAL, Cabinet Office, the Ministry
of Finance, Agricultural Training Institutions, EU Delegation, National Authorising Officer
(NAO), Non -Governmental Organisations (NGOs), farmers representatives/organisations,
Cooperating Partners (CPs) and the private sector.
So far only two PSC meetings have taken place, one per year. This is contrary to the FA12
, which
stipulates it should meet at least quarterly. With only one yearly meeting it cannot be expected
that the PSC oversees and validates implementation as is specified in the FA. Judging by the
minutes, the PSC meetings are mainly informative meetings where the TA TL reports on
implementation and explains the planning process of the next PE.
Change Management Team and Provincial Change Management Teams
The CMT is chaired by the director of Policy and Planning at MAL, to ensure implementation of
PEP activities on a day-to-day basis. Reporting to the PEP PSC, the CMT has central
responsibility for the implementation of the PEP. It comprises ten MAL Directors and
representatives from the Ministry of Finance and the Cabinet Office. The CMT is the senior
decision making and executive body for the PEP and meets quarterly.
According to its ToR, the CMT (assisted by the Secretariat) is responsible for:
Decision making, including approval of PEP activities and budgets
Ensuring the integration of change management processes into the implementation of the
agreed PEP operational plans
The preparation and implementation of a change management strategy
Ensuring the establishment of improved management systems of acceptable international
standards, including changes in the organisational structures where relevant
Appointing PEP Focal Point Persons (minimum one per Department)
Overseeing implementation of a PEP information and communication plan
Receiving reports and prioritising requests from the Provincial Change Management
Teams (PCMTs) and guiding the operations of the PCMTs
Monitoring the implementation process and ensuring adequate reporting activities and
recording of achieved results
Dissemination of results achieved and lessons learned to relevant partners and
stakeholders.
12
TA Team Leader responded that the PSC arrangements as described in the FA were impractical and had to be
amended. This was agreed early on with the Imprest Administrator. The PSC are led through the draft PE and
comment on it. Their recommendations are incorporated into the draft document
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 23 of 65
However, the CMT appears to be more of an informative and coordination unit at the
headquarters rather than the change champion with sustainable mechanisms to institutionalise the
change across the various levels of MAL through the PACOs, DACOs, down to block and camp
level. This means that at levels beyond the national office, the real impact of the CMT is hardly
felt. The big question then is how this can affect service delivery at farmer level. Based on the
one-to-one interviews conducted during the MTE, the very concept of CMT is not known outside
the MAL head quarters.
At provincial level, Provincial CMTs (PCMTs) were expected to assume responsibility for the
local implementation of the PEP, particularly in the area of planning and M&E. The ToR for the
PCMT speaks of a “change management process” that will ensure “efficient and prompt
handling of all the identified changes required to effectively implement PEP”. Reportedly,
Provincial CMTs (PCMTs) were already set up in all provinces during SUPE, but thus far, two
years later, they have met once, but have not been actively involved in implementation. None of
the PACOs and DACOs interviewed by the mission were familiar with the CMT or PCMT
concept. The programme design obviously, and rightly, attributed responsibility for change to the
lower levels of administration. It is unfortunate that the change process did not (yet) reach that
level. For effective change management, it is important to understand the change management
model and look at change as a process with various steps that need to be managed (see Annex 5).
Activity 1.2: Promote Strategic Leadership and Management Development, through
organisational development training courses and orientation workshops, study tours and twinning
arrangements in the region
This activity comprises leadership, management and supervisory training. In both PE1 and PE2
study tours were planned, but subsequently removed. Capacity building for HR staff in MAL
was intended to enable HR practitioners in the ministry to establish a work culture that inspires
the employees in order to contribute to the continuous improvement in the service delivery
provided by the MAL to its clients.
Leadership Training
Leadership training was planned for in PE1 and implemented in PE2. The course was conducted
in Livingstone at a cost of MZW 463,200. Forty-six top management staff members attended in
two batches of 23 each. This was done between 18 and 25 June 2014 by MCA Training
International. A training needs assessment (TNA) was done and an evaluation was done at the
end of the workshop. However the effect of this training cannot be confidently measured as yet
since the training happened only two months ago.
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 24 of 65
All respondents were very appreciative of this meetings and reported that the training contributed
to:
i. An improvement in interpersonal, communication and HR skills
ii. A better understanding on how top management can inspire, coach and have better
interaction with their subordinates
iii. A better understanding of the effectiveness of regular meetings where feedback is
given and information is communicated.
Management Training
With the implementation of the SNDP, the Department of Agriculture has been allocating some
funds for the rehabilitation of infrastructure and equipment that had been neglected for some
time. This is necessary to provide the department and other relevant institutions with a conducive
environment for conducting practical trainings effectively. However it is also critical to have the
necessary human resources with skills and knowledge to effectively manage those training
institutes/centres.
It is against this background that the Department of Agriculture developed a plan to conduct
training of Principals and Farm Managers for all the Farm Institutes and Farmer Training Centres
in order to improve not only business management skills but also their ability to generate income
to contribute to operational expenses. Management guidelines on how to establish demonstration
plots for the priority crops for practical training were also developed.
The Department of Agriculture, with financial and technical support from PEP and the JICA
funded Rural Extension Service Capacity Advancement Project (RESCAP), conducted a
Management workshop for all nine Principals of the Farm Institutes and the Principal of Mpika
Farmer Training Centre from Muchinga Province from 17 to 18 December 2013 at the Masaiti
Farm Institute in the Copperbelt Province. The Principal of the Mpika FTC was included as a
participant in consideration for the future upgrading of the Mpika FTC to the Farm Institute. A
report on the management training workshop for principals (20 principles for 2 days) of farm
institutes is available.
Another training was conducted for middle level managers from across the ten provinces and HQ
between 28 August 2013 and 30 January 2014. A total of 173 members (DACOs and provincial
department heads form all provinces in 8x3-day courses) attended out of a potential 185. The
trainings were conducted in different venues across the country. This was on a budget of MZW
487,293.
During one-on-one interviews, it was reported that the middle level management that had been
promoted from a technical background were empowered on managerial skills.
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 25 of 65
Supervisory Skills training
This training was planned for MAL district officers. The training was conducted at the ZCA,
Monze between 13 and 22 January 2014 at a budget of MZW 94,345. An estimated number of
over 100 officers were targeted.
Based on the evaluation by the TA HR expert, who was present during the training, it appeared
that, although the scope of topics presented was appropriate, with a few exceptions omitted, the
presentations lacked in certain instances theoretical and practical depth that would have made a
difference on the quality front. For example, the presentation on the roles of leaders, managers
and supervisors could have gone deeper to discern the most critical departure points from one to
the other. The focus on leadership should have been on features and traits required of a leader
and what leading really is (setting missions, visions, strategies, values and initiating change in
organisations). Similarly, the role of managers could have covered what managers actually do
(such as systematic implementation of change programmes and strategic plans to achieve desired
objectives by using instruments such as, planning, organising, controlling, performance
management, review and monitoring, etc). Supervision could have taken a greater proportion of
the presentation time and should have focussed more on what and how supervisors perform their
roles (managing performance and development, coaching, providing and receiving feedback,
etc.). In other words, more focus could have been placed on the specific supervisor tasks,
including role profiling and motivation.
The presentation on motivation could have covered more detail on how employees are motivated
and how supervisors can leverage those motivational factors once identified to improve
performance, including (i) the difference between exceptional performance (doing more than
their agreed work targets) and ordinary employees and (ii) how can supervisors instil a high
performance culture and attitudes in their staff? The end of training comments indicated that the
topics like interpersonal skills and performance management were omitted. The training also
lacked in depth both practically and theoretically.
This then questions the integrity of the content and methodology used in the training.13
Activity 1.3: Carry out a comprehensive Services Analysis exercise (focused on districts and
stakeholders of the agricultural sector), as an input for a Functional Review within MAL aimed
at renewing roles and streamlining Ministries systems toward improved organisational efficiency
and effectiveness.
The comprehensive Services Analysis will focus on the districts and stakeholders of the
agricultural sector and will serve as an input for the Functional Review of MAL. The OVI is the
number of studies that have been performed and of which reports are available, which is not a
suitable indicator at result level because these should rather indicate how the reports were used
13
TA disagreed with this conclusion, stating that the comments were meant to help the training providers change the
scope, content and presentation of the programme after this first programme was conducted, and that indeed the
training providers did review the programme for subsequent courses
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 26 of 65
In SUPE a Services Enhancement Review was done. The second quarterly report of PE1
explains how the subsequent Services Analysis can only be done upon approval of the Strategic
Plan.
The formulation of the Strategic Plan 2014-2016 was facilitated during the very first quarter of
PEP implementation, under result area 2. However, the Plan has not yet been approved by the
Minister and is only available in draft. The services analysis preceded the formulation of the
Strategic Plan. The next step (following approval of the Strategic Plan) is for MAL (with
facilitation from Cabinet office and PEP) to carry out an organisational review (as per GRZ
procedure).
Expansion of activities in this result area in PE1 and PE2
In PE1 and PE2 the activities under this subcomponent with a focus on Services Analysis and
Functional Review, have been expanded to activities in quite another direction. These include
support to the National Extension Service Strategy Forum, the Strengthening of Regional
Research Stations (to perform Soil, Seed, Fertiliser and Feed Analysis), the Design and Sourcing
of Resource Packs for Camp Extension Officers (800 CEOs will be equipped with a measuring
tape, soil tester, sprayer and protective clothing) and support to the JICA funded RESCAP
programme in order to support harmonisation of extension messages. A tender process for the
procurement, budgeted at MZW 2,054,000, is currently ongoing. It is expected that the
equipment will enhance the level of financial self-sufficiency of the concerned institutions. This
is not evident, as government institutions have recently been instructed to lodge all revenues in
the general GRZ account. As a result, some institutions (even some that were previously solvent)
are currently suffering from budgetary constraints.
This expansion of activities, clearly outside the scope of the indicated result area, demonstrates
that the programme was able to adapt to the changing circumstances and to display flexibility. It
can indeed be assumed that harmonised extension messages provided by well-equipped
extension officers, guided by scientific data on, for instance, soil analysis and the latest findings
in research, will be able to perform better in service delivery. On the other hand, it should also
be noted that the interpretation of this activity is very curious indeed. In the programme logic the
functional review would be a key element to bring the required change to MAL. It is peculiar
that, rather than introducing change processes, PEP mainly invests in equipment under this key
activity.)
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 27 of 65
3.4 Result Area 2: Sector policy, planning and financial management improved
Table 3.6 summarises the main parameters of Result Area 2.
Table 3.6 Indicators and activities of PEP Result Area 2: Sector Policy, Planning and Financial
Management improved Indicator Sources of verification Baseline Target
Number of policies / legislation
reviewed or developed.
Policy / legislation docs; MAL
annual report
N/A 5
Number of evidence-based policy
advisory notes prepared annually
Policy notes / MAL annual
report
1 2 per year
Number of Policy Dialogue
meetings held annually at
National and Provincial and
District level.
Minutes of dialogue meetings 2 4
Number of SAG meetings per
year
Minutes of SAG meetings 1 to 4 per year 4 per year
MAL budget expenditure as % of
budget released
IFMIS / MAL annual report TBD 10% increase
% of departmental and district /
provincial plans that are
structured in line with the
framework of the Strategic Plan
District and provincial plans N/A 80%
Activities 2.1 Strengthen Sector Dialogue, by strengthening the Agriculture Consultative Forum and
the Agriculture Sector Advisory Group (SAG).
2.2 Strengthen Policy Analysis Capacity, notably by developing data collection,
processing, analysis and dissemination, in coordination with the Central Statistics Office.
2.3 Carry Out Public expenditure tracking Survey and a quantitative Service Delivery
Survey (taking into account recent studies carried out by the WB).
2.4 Enhance Activity Based Medium Term Expenditure Framework and Develop Annual
Budget Processes, by establishing SMART indicators linked to MACO/MLF outcomes
and preparing models for presentation of budgets, in close coordination with the Ministry
of Finance (particularly with PEMFA).
2.5 Enhance Financial Systems and Support the shift to a double entry accounting system
in close partnership with the Finland/ PLARDII programme and in coordination with other
CPs currently operating double entry accounting systems in their programmes with
MACO and line with IFMIS.
Source: Revised PEP Logical Framework (October 2013)
Activity 2.1: Strengthen Sector Dialogue, by strengthening the Agriculture Consultative Forum
and the Agriculture Sector Advisory Group (Ag-SAG)
The first of eight activities in SUPE was to facilitate partnership with Stakeholders and CPs via
supporting Ag-SAG, policy and PEP PSC. One Ag-SAG meeting was supported under SUPE. In
PE1 a large stakeholder workshop was supported to review and endorse the NAIP. This was
funded as an extraordinary Ag-SAG meeting and depleted the entire budget for the activity. The
NAIP was successfully launched in April 2013.
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 28 of 65
Other sub-activities under this activity are support to PACO and DACO meetings, strengthening
DAC and production of policy brief. The PACO, DACO and District Agricultural Committees
(DAC) meetings have not yet taken place. Combined with the non-functioning of the PCMTs,
this is an indication of the challenge of PEP to impact at lower levels of administration.
A policy brief was produced on the proposed changes to the FISP programme: “The Assessment
of Small-scale Farmers’ Perception on the Farmer Input Support Programme Reforms”.
Activity 2.2: Strengthen Policy Analysis Capacity, notably by developing data collection,
processing, analysis and dissemination, in coordination with the Central Statistics Office
This activity is subdivided in (i) policy analysis, evaluation and monitoring, (ii) harmonisation of
polices and plans, (iii) review of the sector strategic plan, (iv) policy impact assessment and (v)
strengthening of the farmers register.
Policy analysis, evaluation and monitoring
In PE1 an analysis of the key policies and plans for the sector was done as a starting point for
policy harmonisation. Also a training needs assessment in policy development, monitoring and
analysis was done.
Harmonisation of polices and plans
As described in the agricultural policy context (Section 2.1 above) there are many strategy and
plans in place, but it is not clear how these documents relate in terms of the hierarchy and key
priorities. A process has started, through a study conducted by the Policy Monitoring and
Research Centre (PMRC), to analyse the various documents and clarify the priority areas and the
relationship between them.
Review of the sector strategic plan
As described under Activity 1.3 PEP funds were used to assist in the production of a sector
strategic plan. The plan has been approved at various levels and awaits formal launch by the
Minister.
In 2014 MAL started the development of an Agricultural Transformation Strategy Policy. The
thrust of the transformation process involves a shift from low-productivity, subsistence farming
to high-productivity agriculture which focuses on agricultural diversification and proposed eight
key strategies. PEP financed two workshops in the process to develop the Transformation
Strategy.
The Strategic Plan (SP) expired in 2010, so a new SP was long overdue. The Ministry, by the
support of PEP, has therefore developed a Strategic Plan for the period 2014 – 2016, taking into
account the dynamics and changes that have taken place in the sector since the last Strategic
Plan. The Ministry Strategic Plan 2014 -2016, aims at increasing the Agricultural GDP growth
from 7 percent to 10 percent. It is expected that the interventions proposed in the Strategic Plan
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 29 of 65
will not only increase the sector’s GDP growth rate, but also ensure food and nutrition security
for the country, facilitate job creation and ultimately contribute to poverty reduction.
Furthermore, the Strategic Plan once implemented, will revamp and sustain growth of the
agricultural sector and make Zambia the regional food basket.
The core strategic issues for consideration in the 2014 – 2016 Strategic Plan were:
1. Inefficiencies and distortions in agricultural marketing;
2. Skewed allocation of resources towards Farmer Input Support Programme (FISP) and
the Food Reserve Agency (FRA);
3. Low agricultural productivity;
. Illegal fishing practices; and
5. Erosion of indigenous livestock and plant genetic resource.
The SP further identifies five main strategic issues. The twelve objectives for the whole MAL are
accompanied with short implementation strategies and several indicators. Its suitability is
demonstrated by the fact that the Strategy is widely used within MAL, for instance during the
ongoing budget preparations. Unfortunately, the SP is only available in draft and has still not
been formally approved
The Human Resource and Administration department has not yet developed its own strategic
plan.
Policy impact assessment
So far only a policy brief on FISP has been produced (see under Activity 2.1 above).
Farmers register
PEP has been supporting the establishment of a Farmers Register in 26 districts, ten in PE1 and
16 in PE2. It is expected to lead to more effective targeting of MAL support. PEP provided
support for every step of the process, such as training of the CEOs, fuel and transportation
allowances, printing of the questionnaires. The paper questionnaires had to be manually inserted
on locally available computers; in some instances CEOs used their personal laptops.
The currently available register is especially used to guide the distribution of the inputs under the
FISP programme, whose targeting issues are known. In theory the register could be used to
monitor whether indeed the smallholders benefit from the FISP and not the larger farmers.
Unfortunately that is not being done at the moment. CEOs also use the register as a record of
their clients but for district planning exercises it is not (yet) being used.
It is estimated that in the concerned districts around 85%-90% of all farmers have been
registered. Farmers are usually eager to register as it will facilitate access to the FISP inputs.
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 30 of 65
GRZ/MAL has indicated their intention to start using electronic systems such as e-vouchers for
the FISP programme. An FAO implemented, EU funded, pilot programme on e-vouchers did not
yet lead to definite conclusions on the methodology to be used and this requires further study.
PEP initiated conversion to a tabled-based farmer register. The JICA funded RESCAP
programme was involved in the discussions as they are operating within the Department of
Agriculture and gave their views.
In each of the piloted Camps visited by the mission (Malende and Choonga Camps in Kalomo
District) about 1,600 farmers were registered. The farmers register appears to be a useful tool in
terms of the following:
It has fostered easy planning and management of farmers
Distribution based on Farmers Input Support Programme (FISP) became easier since it
was easier to identify real farmers who qualified for the inputs
Small scale farmers can now be identified
This cleaned up the previous village register that had charcoal dealers and other traders
who are not farmers
It is easier to get the exact location of the farmer
Both crops and livestock are included in this register. It is more detailed
Improved record keeping
A key challenge is the accuracy of the information in the farmers register. One camp includes
generally more than 1,000 farmers, so it seems unlikely that the CEO will have the time and
resources to verify all provided data. It was observed that in certain households more than one
family member was registered against the same piece of land, so that the acreage given in the
farmers register is far higher than the actual area of the districts. Noted also were inconsistencies
in the information given by farmers, possibly induced by the eagerness of the farmers to benefit
from FISP inputs. This then challenges the integrity of the farmers register14
.
Effectiveness as a preparatory exercise for implementation of the e-voucher scheme is not
evident, as long at the methodology for the e-voucher has not been decided upon and the
requirements of the system to be used (re. necessary data, software used) is not known15
.
Activity 2.3: Carry Out Public expenditure tracking Survey and a quantitative Service Delivery
Survey (taking into account recent studies carried out by the WB).
The Public Expenditure Tracking Survey and Quantitative Service Delivery Survey that were
promoted by the World Bank and included during formulation are not implemented.
14
TA Team Leader responded that these issues were identified in PE1 and that the revised Farmer Register
questionnaire format under PE2 has overcome these deficiencies. However, this is not what the mission observed in
the field. 15
Although the system being used provides data in a format than be converted into any alternative database format
used in the future
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 31 of 65
Activity 2.4: Enhance Activity Based Medium Term Expenditure Framework and Develop
Annual Budget Processes, by establishing SMART indicators linked to MACO/MLF outcomes
and preparing models for presentation of budgets, in close coordination with the Ministry of
Finance (particularly with PEMFA).
In PE1 a review of the MAL planning and budgeting process was done in six districts in
Copperbelt and Southern Provinces. The situation analysis described the current planning and
budgeting process as ‘haphazard, without the active participation of key stakeholders’. MAL
then developed Planning and Budgeting Guidelines to provide a common framework to guide all
levels of the sector to develop Work Plans and Budgets. Next, Resource Allocation Formulas are
being developed to assure equitable resource allocations to PACOs and DACOs. This process is
not yet concluded.
Activity 2.5: Enhance Financial Systems and Support the shift to a double entry accounting
system in close partnership with the Finland/ PLARDII programme and in coordination with
other CPs currently operating double entry accounting systems in their programmes with MACO
and line with IFMIS
The activity started in SUPE with support to the roll-out of the Integrated Financial Management
Information System (IFMIS). MAL employees that need to use IFMIS were identified and
trained. According to the final report of SUPE the implementation of the initial training was not
very effective, due to the mode of training being too theoretical and the heavy demands on senior
staff time.
After some initial resistance to install and use IFMIS, it has now been fully embedded in MAL
HQ and is operational. Reportedly the system still suffers of poor functionality of the ICT
systems at MAL HQ.
Again, rolling out to Provinces and Districts has not been done (yet), although the Ministry of
Finance has already instructed all line ministries to use the system. Also to be completed remains
a review of the intended benefits of using IFMIS, such as a reduction in misapplications and
abuses of Public funds outside planned activities.
The planned financial training at provincial and district level has not yet taken place.
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 32 of 65
3.5 Result Area 3: Human resource management and ICT improved
Table 3.7 summarises the main parameters of Result Area 3.
Table 3.7 Indicators and activities of PEP Result Area 3: Human Resources Management and
ICT improved Indicator Sources of verification Baseline Target
Payroll cleaned up for retired staff Payroll N/A Completed by
2nd year
Outstanding payments of
retirements packages for
MACO/MLF retirees made
Payment records N/A Completed by
2nd year
The number of curricula revised Curricula 0 TBD
% of new staff undergoing
orientation training
MAL annual report TBD 75% of newly
recruited staff
each year
% of existing staff undergoing in-
service training
MAL annual report TBD TBD
HRIS designed HR reports N/A HRIS design
approved
% of staff requiring computer that
have access to a working
computer
PEP reports TBD Baseline +
20%
% of staff requiring internet
access that have working internet
connection
PEP reports TBD Baseline +
20%
Number of departments regularly
updating their website module
Website usage records 0 5 (out of 10)
Activities 3.1 Reconcile Staff Payroll and Make Outstanding Payments of retirement packages for
MAL retirees.
3.2 Develop and Implement HR Strategy by designing and implementing a personnel
records system; reviewing job descriptions; right-sizing staff (in partnership with the WB
right sizing programme); attending to immediate staff development priorities and linking
them to the annual performance appraisal system; developing and Implementing In-service
Training Strategy.
3.3 Enhance Inter-ministerial Collaboration in the area of HR management, in particular
with cabinet Office and MoFNP.
3.4 Improve Planning and Management of Agricultural Education and Training
Institutions.
3.5 Develop and Improve ICT at all levels (including MACO & MLF Web-sites), by
establishing an ICT unit and developing an investment programme.
3.6 Enhance Response to HIV/Aids.
Source: Revised PEP Logical Framework (October 2013)
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 33 of 65
Activity 3.1: Reconcile Staff Payroll and Make Outstanding Payments of retirement packages for
MAL retirees.
During formulation of PEP it was found that there was a large backlog of Repatriation
Allowance, Leave Terminal Bonus and Long Service Bonus (together referred to as outstanding
personal emoluments) of MAL staff. The backlog reduced the budget for agricultural service
activities, prevented some people from vacating their posts and prevented vacation of staff
houses.
Preparation of the payment of outstanding personal emoluments started during SUPE, with a
workshop in October 2012 to review and reconcile PMEC payroll and retirement information in
personal files and audit the outstanding amounts. The workshop and follow-up research resulted
in a list of about 700 persons owing ZMW 17,870,360. The cut-off date was August 2012. A list
of all people on the audited list by province, with amounts adding up to ZMW 17,870,360 was
presented by the PS of MAL in March 2013.
The PEP budget for this activity was Euro 1.12 million, about ZMW 7.7 million in 2013. Since
the PEP activity budget was insufficient to cover the total audited list, and because it was
expected that the total amount of the arrears could rise as other staff might present their files,
GRZ allocated ZMW 52 million in its 2013 budget to pay for outstanding personal emoluments
in MAL. The figure of ZMW 52 million was based on a study during the PEP SUPE aimed at
quantifying all outstanding staff payments (including terminal benefits) at that time.
Before the contribution by GRZ was known, it had been planned to use the PEP funds to give
preference to extension workers and other front-line staff of MAL. Since the combined funds of
GRZ and EU were sufficient to pay for the entire audited list it was decided that PEP funds
would be allocated to Southern Province and part of Eastern Province, with GRZ covering the
remaining provinces and HQ.
By Circular 01 of 2013, dated 22 April 2013, the PS of MAL informed Provincial Agricultural
Coordinators (PACOs) that MZW 12,511,534 had been credited to provincial accounts for
payments according to the audited list. The amounts were as calculated in the audited list except
for MAL headquarters, for which MZW 2,673,228 was reserved against a value of MZW
331,048,638 in the list. The Circular further stated that a total of MZW 7,701,000 was to be paid
by the EU, covering Southern Province and part of Eastern Province.
Further audit and verification of the audited list for Southern and Eastern Provinces resulted in a
final list for PEP payment of MZW 7,032,294 to a total of 274 people (May 2013). According to
the progress report MZW 7,146,197 was paid to a total of 290 people in PE1. In PE2 this was
slightly increased to MZK 7,579,484 to 295 people. Herewith the process is complete and an
audit is on-going.
Information collected in Southern Province showed details of the 185 people who were identified
during the initial exercise of late 2012. A total of 178 people were paid MZW 4,790,855.
Apparently 7 persons did not pass the audit of May 2013. Payment was MZW 567,965 below the
figure in the audited list.
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 34 of 65
Interviews with stakeholders at different levels give a mixed appreciation of this activity.
Obviously those whose outstanding payments were made are most happy with it. In summary the
following observations can be made on the activity:
The whole process of identifying the outstanding payments was cumbersome. Personal
files had to be collected from all over the country and moved up and down. The
weaknesses of the HR information and management systems became evident
Nevertheless the activity was implemented in a transparent manner and is well
documented
The fact that GRZ allocated ample funds to this activity, surpassing the amount allocated
by the EU, shows that PEP acted as a catalyst to solve this outstanding business
Outstanding personal emoluments has been an ongoing issue, bothering MAL staff at
different levels. The payment of arrears reduced the hassle of MAL managing staff
It was reported that payment of arrears resulted in retired staff vacating staff houses to the
benefit of their successors. This was explicitly reported at ZARI. At Camp level houses
may have been vacated, but often in such poor state that new staff refused to occupy them
There was no link found between the payment outstanding personal emoluments and the
recruitment of new staff. Staff has been recruited regardless of outstanding payments
The payment by the project (and GRZ) only affected staff retired up to 2012. Outstanding
payments to staff not yet retired were not included, nor payment to staff retiring after
2012. Therefore the activity solved the situation for part of the MAL (ex-)staff, not for all
and not for ever. Although as from 2014 the payment of terminal benefits has ceased and
all entitlements are wrapped up in recurrent staff salaries.
Activity 3.2: Develop and Implement HR Strategy by designing and implementing a personnel
records system; reviewing job descriptions; right-sizing staff (in partnership with the WB right
sizing programme); attending to immediate staff development priorities and linking them to the
annual performance appraisal system; developing and Implementing In-service Training Strategy
An HR strategy does not exist, but is expected soon after the approval of the Strategic Plan. The
overall strategic plan for MAL is what is used despite having not been approved by the minister.
A personnel record system does not exist but it is being worked on as discussed under the HR
database. Job review was last done when the organisational review was done in 2003 as
discussed below on recruitment. The immediate staff development priorities that have been
addressed are the leadership, management, supervisory, (Result 1) HRIS and induction trainings.
These were based on Training Needs Assessment and partly on APAS.
Under this activity, various sub-activities were formulated regarding HR database, capacity
building of HR staff, organisational review, in-service training and induction/orientation training.
HR database
The Internet is essentially a summation of millions of computers interlinked together comprising
a massive and elaborate virtual database system. Its power is in its search and query capabilities
enabling end users the ability through search engines to filter out desired information within
fractions of seconds. The value of a methodical and structured way of storing and accessing
information is without a doubt a critical component of performance enhancement in any
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 35 of 65
organisation. Searching and attempting to piece together fragmented information or shuffling
through manual records can be quite crippling.
MAL has an opportunity to begin to convert manual records into an electronic format. There is a
need to harmonise and consolidate data from various departments and external sources. This will
help enhance strategic initiatives for the agricultural sector. Zambia can be a viable agro based
nation with a strong competitive advantage in the Sub-Saharan region of Africa, and the use of
ICTs will play a pivotal role. The goal of a comprehensive integrated database structure for MAL
is to provide timely, usable, accurate, and relevant data to MAL decision makers and
stakeholders in order to increase effectiveness of both framing and implementing policy.
A feasibility study for the development of a HR database was undertaken during PE1 and the
system was to be installed in PE2. Among the PE2 planned activities was the establishment of
the HRIS in MAL as part of an effort to improve the performance of HRM and ICT in the
Ministry. The existing Payroll Management and Establishment Control (PMEC) was to be
extended to provide other required human resource information. The staff who were already
trained to use PMEC were re-trained under PEP to facilitate this integration. The payroll needed
to be reconciled, e.g. when one staff moved from one station to another, the information would
remain in the previous station. There were no records to indicate who was where, skills level and
what the qualifications were. So far 15 staff have been re-trained on the merging of PMEC and
HRIS at a budget of MZW 92,890 at the Provincial Administration Head Office in Ndola.
Currently PEP is helping in the process of designing a form/questionnaire to collect data of
important information on the current employees and their qualifications within the ministry.
PMEC so far is the only source of HR information but this seems outdated based on the current
need of accurate and precise information on employees16
. The target is to complete the merging
exercise by December 201417
.
Recruitment
This is an ongoing process by the Public Service Commission. Every job has a job description
that was developed once the last organisational structure was developed in 2003. Despite the
changes in the working environment, the job descriptions have not been reviewed since. The
recruitment is based on the establishment and not necessarily on the number of retirees. In some
cases those recruited were more than what the establishment required and in some cases less. The
number recruited was not equivalent to that of those retired. Based on the data collected during
the one-on-one interviews, about 75% of the positions have been filled. Currently there is a
freeze on recruitment, a phenomenon that happens every few years. The benefits of new
employees coming on board is that they came with new levels of energy and new technology.
16
TA Team Leader adds: the point is that the PMEC system currently only includes payroll data and is handled by
Cabinet Office. MAL has no electronic HR database and have no control over the payroll programme, consequently
this does not accurately reflect the actual staffing situation. If MAL can gain access to the HR module of the same
programme, they can update staffing records and link this to automatically update the payroll. 17
Development of HR Data base Training Programme; Activity Report -ActivityNo:3.2
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 36 of 65
Capacity building of HR staff
The capacity building that was done is what was discussed in Result 1 and the training on HRIS
discussed in the HR data base section.
Organisational review
Despite the ministry having gone through several changes, there had not been an organisational
review since 2003, when an organisational structure was developed. However, a review process
is ongoing and soon a new organisational structure could be rolled out.
The Annual Performance Appraisal System (APAS)
The APAS is meant to appraise employees in order to enhance an individual’s performance
throughout the year. The APAS is implemented in a transparent manner and this helps officers to
comprehend better inadequacies and competencies.
In-service training
The ten farmer institutes are basically for in-service training for extension staff and also for
provision of higher level training to progressive farmers/lead farmers and commodity
demonstrators. The refresher courses disappeared around 1980, and this led to a reduction of the
knowledge level of the extension officers. Training institutions became too academic with
insufficient practical skill training. Under PEP a TNA has been conducted with a view to
upgrading short-course curricula of the ten Agricultural Training Institutes. Currently a TNA is
ongoing to help establish the skills and knowledge gaps. This will serve as an input for
curriculum development. The curriculum has insufficiently been upgraded to meet the current
market demands. The Zambia College of Agriculture (ZCA) Monze reviews its curriculum every
three-four years. The last one was done in 2011.
Induction/orientation training
Since the inception of the National Institute of Public Administration (NIPA) in 1962, it was
mandatory for all newly recruited civil servants to attend induction and orientation programmes
at the Institute. This was aimed at helping employees know and understand the civil service work
culture, values, government policies, procedures, expectations, terms and conditions of
employment and basic general orders needed for one to settle down quickly and begin to perform
duties as expected by the supervisors. Due to budget constraints, this stopped in mid 1990s.
MAL has subsequently realised that the absence of such induction programmes is
counterproductive, thus the decision to have it re-instated was made. Under PE1 MAL conducted
the training using PEP funds in October 2013 and in July 2014 under PE2 in Eastern and
Western provinces. This was done in Western Province, Mongu at a budget of MZW 205,808
where 40 out of 185 (new employees) participants attended. The second training was also held in
Mongu and 81 participants attended at a budget of MZW 355,003. A handbook to be used for
orientation is being developed.
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 37 of 65
The programme covered the following topics: Mission and Vision of the Ministry; Ministry
information on clients, customers, stakeholders; Responsibilities and overview of the Ministry
and functions of its various Departments; Organisation structure of the Ministry; Structure of
Government; Financial Regulations; Terms and Conditions of Service; Code of Ethics for the
Public Service; Public Service Training and Development Policy; and Human Resource Policies
and Procedures in the Public Service18
.
The purpose of the training was to induct and orient the newly recruited staff into the Public
Service in general and MAL in particular, and prepare them to perform their roles effectively and
to the expectations of the ministry and its clientele and customers. It was reported that this
purpose has indeed been achieved and supervisors expressed their satisfaction with the newly
recruited, and inducted, employees. It seems likely that this activity will be sustained, even after
the completion of PEP.
Activity 3.3: Enhance Inter-ministerial Collaboration in the area of HR management, in
particular with cabinet Office and MoFNP
For performance effectiveness, there is need for ministries concerned with serving various
stakeholders to work in harmony and synergistic modus operandi. From the one-on-one
interviews, it was noted that a level of collaboration exists as explained below:
i. The HR department is working towards integrating PMEC and HRIS with a view to
improving on employee data storage and retrieval. The staff that were trained on IFMIS
have been re-trained on this integration. There has been inter-ministerial collaboration
between the cabinet office and MAL in the sense that currently the cabinet office is
helping MAL in the development of the HRIS.
ii. The department of Food and Nutrition in MAL works in collaboration with the Ministries
of Health and Community Development to address matters on nutrition, malnutrition,
health, balanced diet. The department of Food and Nutrition also works together with
research institutions more particularly on the latest programmes of fortified foods.
iii. There is a positive link between PSMD and the HR secretariat. This is clearly noted
especially during recruitment where HR would recruit lower positions and PSMD recruits
senior positions.
iv. Management Development Division of Cabinet Office facilitated the updating of the
strategic plan and will facilitate the organisational review.
Activity 3.4: Improve Planning and Management of Agricultural Education and Training
Institutions.
MAL has in its mandate training programmes implemented through Agricultural Training
Institutions (ATIs). The overall objective of this intervention is to provide critical mass of
suitably and adequately trained human resource base for both the public and private agricultural
sectors. It is envisaged that such quality human capital will strategically contribute to agricultural
production and productivity in Zambia.
18
Final Programme for Induction Programme for newly recruited employees into the Public service and MAL
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 38 of 65
There are ten ATIs at which training is offered under the auspices of the MAL. Between
December 2013 and February 2014, a situation analysis was conducted on the ten colleges. This
included an assessment of the status of human, physical and fiscal resources at the disposal of the
institutions in carrying out their mandate; the type and scope of training programmes being
offered need to be assessed for their appropriateness and relevance; and the governance of the
institutes, as it creates an environment for the synergistic interactions among the various
components mentioned above and is important for efficient and effective training much needed
human capital.
The MTE team visited Monze College of Agriculture. It was noted that institutional policy
changes recommended in the past have received little attention and their implications are still
valid to date. The team observed that coordination of agricultural training was through two ATI
Coordinators at MAL, but given the diversity of the disciplines encompassing animal health,
nutrition, horticulture, agricultural marketing to general agriculture, in the college such support
cannot come from one office thus renders the support rather weak.
However the coordination of the ATIs needs to be housed in a special Directorate that would
oversee the entire training function on behalf of the MAL. This would improve the integrity of
training as a process.
It was noted that funding for the ATIs was mainly from government with donors contribution
being minimum. While most of the funds were from government, the disbursement was ‘erratic’,
untimely and always below the budgeted amounts. This has made the college find means of
generating additional funds through income generating activities (IGAs). PEP supported in the
purchase of a computer, a printer and a flip chart holder/white board. PEP also allows the
College to generate income by contracting it for the provision of the Supervisory Skills Training.
PEP also has plans to facilitate in curriculum development.
Activity 3.5: Develop and Improve ICT at all levels (including MACO & MLF Web-sites), by
establishing an ICT unit and developing an investment programme
ICT management and training
The employment of various Information and Communication Technology (ICT) tools in any
organisation is critical for information sharing and flows between and among departments. In
sub-Saharan Africa, where agriculture is the main source of livelihood, information sharing on
new agricultural technologies, a crop and rain forecast, among other important aspects of
agriculture is important to achieve improved yields. One of the most effective tools and
platforms to share such information is through a functional website.
MAL engaged the EU’s support for PEP to renovate and upgrade its website. A functional MAL
website would significantly improve access to and visibility of MAL activities, services and
products. Stakeholders such as farmers, seed growers, agricultural research institutes, colleges
and investors can also easily access this information. Further, the website also provides a
platform for MAL research staff to share their findings. For example, this would be useful for
agricultural students conducting research and agricultural institutions formulating agricultural
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 39 of 65
policy. An upgraded MAL website is essential for achieving an agricultural sector which is
competitive and knowledge based.
The methods and approach employed in the upgrade and renovation of the MAL website
included a review of the MAL website which included, collecting information through
interviews, focus group discussions and surveys across MAL levels (Head Office, Provincial,
District, Block, and Camp). A Web Designer from Golden Greeza Media Consultants was
engaged to re-design the website with the support of key MAL stakeholders. This included
facilitating a Web Content Management training for officers from the National Agriculture
Information Services (NAIS) and developing a MAL Website training manual19
.
The existing ICT infrastructure in MAL was poorly developed, uncoordinated and operating sub-
optimally. At the Ministry there are approximately 250 computers, with around half of these
connected via a local area network to an email and internet server. However the email used to be
frequently down due to overload and there was no net-working. Under PEP, the internet
connectivity was improved and the MAL staff can communicate through internet. The MAL
currently owns a functioning website upon which documents are uploaded albeit with a lot of
difficulty. This was originally developed with support from ADSP, but was not regularly updated
or populated. PEP supported the re-design of the site, training of NAIS staff and is currently re-
locating the web servers from Zamtel to Mulungushi House, so that it can be regularly updated.
Security systems at Zamtel were so rigorous that NAIS staff failed to upload content.
Investment Programme
The agriculture sector is a fundamental part of development in Zambia. If MAL is to contribute
to the overarching goal of moving Zambia to a self-sustaining middle-income country, it needs to
leverage a strong ICT system that will significantly improve its agricultural information flows.
One of the strategic initiatives reflected in the ICT Strategy and Action Plan document (2013-
2015), developed with PEP support during the SUPE, was to assess the ICT needs and
requirements at all the MAL levels of operation, with the view to improve efficiency and
effectiveness of MAL service delivery to clients and stakeholders.
A robust, well maintained ICT infrastructure is a catalyst to transforming internal and external
communication systems. Appropriate ICT tools will equip MAL to have seamless information
flows necessary for responsiveness in addressing agricultural sector needs across the country.
An assessment of current ICT needs and requirements conducted via multiple methods; focus
group discussions, one-on-one interviews and surveys indicated constraints and challenges in the
availability and flow of information at various levels. Subsequently, there is need to invest in
ICT resources including equipment and human capacity at all MAL levels (Head Office,
Provincial, District, Block and Camp). Effective deployment of appropriate ICT technologies
will result in the improvement of the overall performance of the Ministry.
19
Ministry of Agriculture and Livestock (MAL) Website Upgrade and Training Report, January 2013
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 40 of 65
Plans are underway to purchase ICT equipment, hardware, software and licenses. Such items will
include but not limited to;
1. ICT and Productivity Training
2. Laptop (Block Supervisor)
3. Basic Computer training (ICT)
4. Portable Solar Panel (for recharging devices)
5. Airtime- internet bundles (per month)
6. Anti-virus software
7. MS Office Software
8. MS Operating System
9. Third Party support services
10. Flash drives
11. External hard drive
However there is no management staff for the ICT.
Activity 3.6: Enhance Response to HIV/Aids
HIV/AIDS
Evidence points to the fact that, despite the down-ward trend in global HIV infections, Sub-
Saharan Africa (SSA) continues to bear the heaviest burden of the global total. At the end of
2009, for instance, SSA had 22.5 million people, or 67.5 per cent, living with the HIV virus
against the global total of 33.3 million (UNAIDS). Again, out of a total of 1.8 million people
who died of AIDS-related illnesses world-wide in 2009, 1.3 million, or 72 per cent, occurred in
SSA. This is in spite of the reduction in the rate of new annual HIV infections from 2.2 million
in 2001 to 1.8 million in 2009.
MAL is committed to realising the vision and goals outlined in its Agricultural Policy. In order
to achieve this, the ministry is analysing how human resources can be preserved from the
distressing impact of HIV and AIDS. Building on the principals and priorities of the National
HIV and AIDS Response Framework, this policy will serve as a guide to define the sector
operational activities and resource allocation towards HIV/AIDS. The policy has been reviewed
through nationwide consultations, informed by best evidence and driven by moral imperative to
achieve, apart from the above, a National access to HIV and AIDS prevention, treatment, care
and support. MAL is committed to leveraging existing partnerships with staff, communities and
National Champions to support the implementation of this policy (HIV and AIDS and Wellness
Workplace Policy for Agriculture Sector Final Draft, Feb 2014).
Under PE1, PEP supported the updating of the MAL's HIV/AIDS Workplace Policy 2011-2015.
This addressed prevention, testing, mitigation of stigmatisation and activities that reduce risks
based on risk assessment and impact assessment. These were printed and distributed to promote
a built -in response to the situation and ensure greater awareness. There was leadership
sensitisation on HIV/AIDS. Trainers were also trained. This programme potentially utilises the
Ministry's countrywide extension networks as a channel for HIV/AIDS sensitisation campaigns
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 41 of 65
Gender Activities
In the past three decades, gender inequality has been identified as a bottleneck for development
programs as it risks to isolate half of a population, women. It is a root cause for major
programmes to take a course of a blanket approach ignoring the diversified needs and interests of
women and men. This has significantly contributed for women’s disadvantaged position in most
growth indicators. The historical segregation of women roots and intertwines with tradition,
culture, and religion in most parts of the world. This put women compared to men in a
subordinate position and compromises their access to and control over major resources including
opportunities like education which definitely opens door to better life and progress in their future
endeavour (Gender Mainstreaming Guidelines and Checklists for Agricultural Sector (Final
Draft, February 2014 - prepared under PEP).
In Zambia the conventional gender role and socialisation ascribe a lower position for women that
make them subordinate to and dependent on males. This increases the vulnerability of women
and girls. Other factors that have contributed to gender inequality are stereotypes, patriarchal
structures, culture values, scarcity of resources, traditional family roles/responsibilities and low
education and literacy levels.
Addressing vulnerability of women and girls could contribute to improvements for human
development. The government of the Republic of Zambia has made efforts at the national level
to reduce women’s vulnerability. The Government had developed a National Gender Policy and
created an institution to guide the nation on gender and development, using a gender
mainstreaming strategy for making women's as well as men's concerns and experiences an
integral dimension of the design, implementation, monitoring and evaluation of policies and
programs in all political, economic and societal spheres so that women and men benefit20
.
Gender affects the workforce at both government and farm levels. During the Start Up
Programme estimate (SUPE), a situation analysis was done regarding gender issues in MAL and
a strategy was proposed and an action plan developed1. In PE1 gender mainstreaming guidelines
were developed to promote gender equality, responsiveness, create ownership, alignment
harmonisation, mutual accountability and managing of results to ensure equality and equal
opportunity. Top leadership were sensitised. In PE2 a check list was developed and this is to be
used as a hand book on gender mainstreaming. It was further recommended to raise the profiles
of the cross-cutting themes by establishing high level committees composed of Directors, chaired
by the Permanent Secretary, sub-committees composed of specialists from different departments,
and committees at provincial and district levels and at the training institutions. In addition it was
recommended that Focal Points should be designated at all levels and that their job descriptions
should be formalised and included in the Annual Performance Appraisal System (APAS). At the
moment these arrangements have only been partially implemented.
20
National Gender Policy 2000
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 42 of 65
Conclusion on Result Area 3
A wide range of activities was implemented in this result area, of which some are a first step
towards a major improvement (e.g. ICT, Agricultural Strategy, trainings). It is imperative that
the MAL/PEP demonstrate that the subsequent actions are also following through, so that indeed
the MAL operations can become more efficient and transparent. Examples are the Functional
Review, the HR policy and right-sizing of staff, the implementation of the M&E system. If these
will be accomplished during the implementation period of PEP, these can be considered major
achievements.
3.6 Result Area 4: Monitoring and evaluation improved
Table 3.8 summarises the main parameters of Result Area 4.
Table 3.8 Indicators and activities of PEP Result Area 4: Monitoring and Evaluation improved Indicator Sources of verification Baseline Target
% of quarterly M&E reports
generated timely i.e. in line with
M&E system
M&E reports 0 75% (3 out of
4 quarters in
the last PEP
year)
M&E database system operational M&E reports N/A M&E database
fully
operational
Activities 4.1 Review and enhance the M&E System in coordination with IFAD/Finland support, notably by establishing a set of indicators for the sector, sub sectors and service delivery units, as well as developing data reporting systems and reporting formats
Source: Revised PEP Logical Framework (October 2013)
Under this result area two levels of M&E activities have been reported: support to the M&E
systems of MAL (in accordance with the PEP logical framework, and subdivided in two sub-
activities: M&E training at all levels and supporting the establishment of a Zambia SAKSS) and
the development of an M&E framework for PEP.
PEP M&E framework
The development of the M&E framework for PEP took place in PE1, when a short-term
consultant made some improvements to the programme’s logical framework and derived from it
a separate logical framework for the TA. Main changes compared to the original logical
framework were (i) consistency between the logframe and the text in the FA, (ii) improvements
in some indicators and (iii) the definition of targets.
PEP progress reports mention that the revised logical framework is now being used for planning,
monitoring and evaluating programme performance by programme managers (Imprest
Administrator, Imprest Accounting Officer and TA Team Leader), but there are no indications
that this is actually the case. The latest version shared with the mission was dated October 2013.
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 43 of 65
The baseline and/or targets of many of the indicators were still labelled “to be defined”. There is
no reporting against the indicators of the logical framework.21
It was also noted that some activities have been developed in the PEs that don’t really fit in the
LFM. It transpired that activities can just be suggested by MAL for funding through PEP,
regardless of the intervention logic of the Financing Agreement. The TA TL in such cases point
out to the flexibility phrase which is also in the FA. MAL obviously appreciates this kind of
flexibility, as it allows the ministry to use PEP funds to fill any funding gap of other
programmes. It makes, however, monitoring of activities and results against indicators in the
LFM, impossible.
M&E training at all levels
In PE1 PEP supported the completion of the M&E manual for MAL (MAL Monitoring and
Evaluation System, July 2013). The report was thereafter edited by Regional Strategic Analysis
and Knowledge Support System (ReSAKSS; MAL Monitoring and Evaluation Manual, April
2013). The M&E manual is now ready, with its tools and indicators. What remains to be done is
the training of M&E staff at different levels in data collection, entry, analysis and report
generation.
For staff training on M&E, PEP coordinates with IFAD’s (with Finish and Swedish funding)
SAPP (Smallholder Agribusiness Promotion programme). SAPP has scheduled to train
Provincial Planner as Trainers of Trainers (ToTs) in M&E. PEP has made provisions in PE2 to
finance training of District Agribusiness Directors by the Provincial Planners and the rolling out
of M&E systems to all levels as far as Blocks and Camps. The process is held up by the delay in
the recruitment by SAPP to train the ToTs.
Now that a common M&E framework is in place in MAL it remains a challenge to the ministry
to implement it as such. So far different departments and programmes have done their own,
isolated M&E (if at all). Now somebody must take responsibility to implement the
comprehensive system. This responsibility belongs in PPD, but so far there is not a dedicated
position in PPD to concentrate fully on M&E. Also the capacity to collect, analyse, disseminate
and use M&E data is not yet in place.
Establishment of a SAKSS node in MAL
In PE1 PEP supported a workshop by MAL, RESAKSS-SA and the Indaba Agriculture Policy
Research Institute (IAPRI) as part of the process to develop M&E tools. The requirements of the
SAKSS Annual Trends Report (ATR) were also incorporated).
3.7 Cross cutting issues, communication and visibility
HIV/AIDS and gender issues are incorporated in PEP, as described under Activity 3.6 above.
Activities around HIV/AIDS and gender progress slowly. Some consultancies have been done,
e.g. resulting in a gender checklist, but the checklist is not yet printed. Or to have a HIV/AIDS
policy, but the policy is not implemented. The gender focal point is operating rather in isolation. 21
TA Team leader states that the PEP logframe is reported on in each PE Final report
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 44 of 65
Of the funds budgeted by MAL for gender activities (MZW 567,000), only 25% was released to
date.
A PEP Communication and Visibility Plan was drafted in PE2. Stickers have been designed and
printed for the two project cars, which were delivered at the beginning of PE1, and for smaller
items such as training aids and computer equipment. To the MTE team PEP, or EU, was not
generally visible. The TA team has no office in MAL, so can’t indicate its presence. No
reference to PEP or EU was seen in the offices of PPD, or anywhere in the visited provinces and
districts.
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 45 of 65
4. Conclusions
4.1 Relevance
Support to the Agricultural Sector was and is still relevant. Important strategic documents, such
as Vision 2030 and the FNDP, the 9th
and 10th
EDF CSP, all highlight the importance of this
sector. The SNDP (2011-2015), developed during the implementation period of PEP, features the
theme ‘’Sustained economic growth and reduced poverty” and agriculture has been identified as
the main priority sector in achieving both economic growth and reduced poverty.
Poverty amongst the rural population is high at 77%, with a stunting prevalence of 45% - of
which 21% severe. Food insecurity stands at 42%, with the average number of months without
food being 3.2 months22. The NAIP 2014-2019 attributes this high levels of poverty and food
insecurity the following factors:
The small size of the holdings (73% of all small-scale households cultivates less than 2
ha of crops), caused by both labour and capital constraints, and, increasingly, limited
availability of land
Low yields, which are significantly below global averages
Inability to take full advantage of government support measures in input supply and
marketing
Programmes funded by the EU in the Agricultural sector prior to 2009 had indicated that MAL
(at that time still MACO) was facing “serious limitations with regard to its ability to fully assume
leadership of the sector as well as to deliver quality extension services to the farmers”23. This
observation formed the basis for the design of the current PEP programme.
The (draft) Strategic Plan 2014-2016 for MAL that was prepared in 2013 identifies the following
weaknesses of MAL: (i) Weak Human Resources Management, (ii) Weak synergies between
departments, (iii) Limited use and application of ICTs and, (iv) Fragmented M&E system. All
four areas are currently addressed by PEP, which illustrates its continued relevance till date.
The environment in which the MAL operates is indeed continuously changing, for instance the
most recent policy documents (NAP, draft Dec 2013; the Strategic Plan, 2013) demonstrate a
strategic move towards a facilitating and regulating government, with an increasing role to play
for the private sector. The recent establishment of the Scaling Up Nutrition (SUN) secretariat in
the office of the Vice-President is a result of the growing realisation that Food and Nutrition
Security (FNS) cannot be attained by increasing productivity alone. This concept still needs to
be fully adopted by MAL, which will include changes in attitudes and practice.
A programme to support MAL into developing into a more flexible and open-minded
organisation is therefore certainly useful. This was also confirmed by the MAL Permanent
Secretary and CMT members.
22
National Agriculture Investment Plan 2014-2018, CAADP, Ministry of Agriculture and Livestock, GRZ 23
Action Fiche Support to the Agricultural Sector Performance Enhancement Programme, 2009
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 46 of 65
At the same time, it should be recognised that if a major expected outcome is to reduce poverty
and improve food and nutrition security of rural households, all efforts should concentrate on
creating an enabling environment in which the smallholder can improve productivity and engage
in income generating activities. Improved GRZ extension services may only constitute a small
factor in the smallholders’ decision making process and other factors may be more important
(availability of affordable inputs, access to market, land). Indeed, the MAL budget allocations to
FISP and FRA suggest that these factors are already considered more important by the Zambian
Government.
Shortcomings were encountered in the intervention logic. As described in Section 3.2 this
problem stems already from the problem analysis at the conception stage of the programme. The
Programme Purpose and Results are not well defined. Some of these shortcomings were reported
by two ROM missions prior to the MTE, but were not adequately addressed. As a result PEP
after more than three years of implementation doesn’t have an functional LFM and indicators are
not up to standard, or not measured, to allow monitoring and evaluation. Activities in PEs are not
always planned in the context of the programme logic, resulting in isolated actions, not
contributing to the programme purpose.
Another challenge concerns the overall change process, which is supposed to form the basis of
the programme. Result 1 of PEP serves to support the change process in MAL. For the analysis
of the change process reference is made to Annex 5, which summarises the logical steps in a
typical change process. Looking at the model, steps 1 and 2 were made in PEP: there is
awareness about the need for change (to improve MAL services) and a CMT was set up to
manage the change process. Then various activities are being implemented, but this is step 5 of
the model. Step 3 (development of a vision & short term targets) and step 4 (communicate the
vision/targets) were not made, which results in activities not automatically contributing to the
desired situation.
Cross-cutting issues, like HIV/AIDS and gender are incorporated in PEP in the form of two
specific activities. They are not mainstreamed in all activities of PEP, e.g. in the selection of
beneficiaries.
Conclusions on relevance:
• PEP was and is a relevant programme in the sector and policy context of Zambia
• The design of PEP shows shortcomings in terms of intervention logic and indicators
• There are no suitable indicators to measure achievement outputs and Programme Purpose
• Although poverty alleviation is explicit in GRZ and EU policies, PEP does not
specifically target poorest
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 47 of 65
4.2 Efficiency
After serious delays in programme implementation, at least in SUPE and PE1, the list of outputs
achieved has been growing in the course of PE2. Main achievements can be listed as follows:
• Leadership, Management and Supervisory Training;
• Strengthening of extension co-ordination; extension harmonisation guidelines completed;
extension strategy almost complete;
• Updating of farmer register updated in 26 districts and piloting of a computerised process;
• New Strategic Plan developed;
• Development of MAL Planning and Budgeting Guidelines;
• IFMIS training facilitated;
• The clearing of personal emoluments for 300 retired staff;
• Human resource management training for Senior/Executive Officers;
• The development of a Human Resource Information System;
• An induction training programme developed and piloted;
• Training needs assessment to identify priority in-service training needs; management
training for all senior agricultural officers; management training for all FTI principals;
• Website up-graded, email system up-grade under way;
• HIV & AIDS Wellness and Workplace Policy developed;
• Gender Mainstreaming Guidelines developed;
• MAL M&E manual completed;
• Support for the Strategic Analysis and Knowledge Support System (SAKSS) Annual Trends
Report (ATR)
Generally these activities have been well implemented, to the satisfaction of direct beneficiaries
and within the budget. However, as described in the section above, the poor application of the
LFM makes this a list of largely isolated activities, of which the contribution to the higher level
results and purpose cannot be assessed. It is also observed that some of the activities which
should target the provincial and district levels have not yet been implemented.
The PEs serve as the main planning instrument and these reflect the outcome of multi-day
planning workshops. As the PEs are originally financing instruments, the emphasis is on
activities and budget lines. The multi-day planning workshops were not sufficiently used to
revise progress against the result areas and did not result in the identification of appropriate
progress indicators against the upcoming work plan.
The absence of a logical sequence of activities results in lists of unstructured and often unrelated
activities, illogically combined under one expected result. As a consequence, implementation is
very activity- and not result-oriented. Due to the absence of key milestones or key progress
indicators, assessment of the actual use of the provided resources can therefore not sufficiently
be monitored.
It was observed that the programme did display great flexibility in altering or revising activities
once a new need materialised. A certain amount of flexibility is generally commendable,
however, in the case of PEP it has also led to a loss of identity and is sometimes considered as an
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 48 of 65
additional pool of funds for MAL and the PEs as a “menu” from which can be picked and
chosen.
The programme has suffered serious delays, especially during the start-up phase, which had to be
remedied by two Riders to the FA.
The PEs are quite long and very descriptive. For instance PE2 contains 13 pages of context and
the budget is extremely detailed, up to the specific DSA for two drivers from a certain district.
This appears to be a not very efficient use of the TAs time. It would be better to limit for instance
the context to the changes in context that have developed since the previous PE. The very
detailed budget suggests a sense of accuracy that does not really exist; for instance, the ICT
equipment as is mentioned in the PE2 does not correspond with what will be procured. This is
not contrary to EU rules and regulations, because these allow flexibility under the main budget
headings.
Payment of Outstanding Personal Emoluments
The payment of outstanding personal emoluments constitutes a large part of the programme
budget. A detailed description of the efficiency of disbursement and the accomplishment of the
desired outputs is provided in Section 3.5 (Activity 3.1). Although the activity was implemented
in an efficient way, given the challenges of MAL’s HRMS, the activity as such is not regarded
appropriate (payment of government staff emoluments) nor efficient (not leading to the expected
result).
Procurement of equipment
Serious delays have occurred with the procurement of equipment, such as the two vehicles for
the TA (more than 16 months). At the time of the present MTE, several tenders were still
outstanding for different lots (extension officer kits, research equipment, computers and laptops).
It is not clear whether the items will arrive sufficiently in time to be able to provide the
accompanying training courses or to monitor the achieved benefits (e.g. from lab equipment).
Extension manuals and guidelines
The JICA funded RESCAP programme supports MAL in the area of extension. The philosophy
behind the programme is that only TA is provided and that all other expenditures will have to be
borne by MAL, in order to create ownership and sustainability. PEP has been working closely
together with RESCAP and has provided funds for the RESCAP organised training activities.
PEP has also provided funding for the printing of the General Operational Guidelines for
Agricultural Service Providers. These Guidelines will be used to harmonise extension messages
by both private and GRZ entities.
Communication between the different actors
It is evident that the TA and the PSU are very diligent and committed to PEP. The efforts of the
TA staff are appreciated by MAL and good working relationships have been established. During
the revision of the LFM, a separate LFM was prepared for the TA, for reporting and monitoring
purposes. This is unusual and demonstrates rather how the TA is not considered as an integral
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 49 of 65
part of the MAL.24
At the same time, it was observed that the TA performs tasks and duties that
go well beyond that what would normally be expected of Technical Assistance.
The management structure of the programme is not following the prescriptions of the Financing
Agreement. PEP reports repeatedly refer to the heavy workload of MAL senior staff, which
prevents a more regular engagement of CMT and PSC. This affects the efficiency of the
programme and contributes to the delegation of many task to the TA, which should be done by
MAL.
The EC Delegation has been actively monitoring the programme and is closely following its
accomplishments. Whereas the positive attitude and the willingness to contribute to the success
of the programme are commendable, a more detached involvement might have led to a stronger
focus on results and key-progress indicators. Accordingly, more emphasis should have been
placed on the recommendations of the ROM missions.
Conclusions on efficiency:
• After serious delays in SUPE and PE1 the programme has gained more momentum
• A good number of activities are completed with good quality output
• Reporting and financial management are up to standards
• Involvement of MAL senior staff is not as intensive as would be expected
• Implementation is activity- and not result-oriented
• Monitoring of implementation is hampered by poor logical framework and indicators; the
LFM is not used for management
• Activities involving decentralised implementation are delayed
4.3 Effectiveness
According to the 2006 Methodological bases for Evaluation of External Assistance, the
definition of effectiveness is “the extent to which the development intervention's objectives were
achieved, or are expected to be achieved, taking into account their relative importance”. It was
argued in earlier sections of this report that the weakness of the logical framework, together with
the definition of activities outside the programme logic, resulted in a break of logic between
activities, results and objectives. In conjunction with the absence of SMART indicators, it
becomes impossible to ascertain to what extent the activities will lead to the achievement of the
objectives. Here the effectiveness of some of the larger of the components is described.
Training programmes
Many training programmes and training courses have been provided by the programme at very
different levels of the Ministry. No entry tests were done nor other data collected to determine
the pre-test situation. At the end of a training, a general evaluation was done on the overall
impression of the provided course, but again no actual tests were taken. At result level, no
suitable indicators have been provided to measure to what extent the provided training was
24
TA Team Leader points out that this also reflects the fact that the TA service contract and PEs have separate
reporting arrangements. In the TA reports they are required to demonstrate how they have contributed to programme
implementation.
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 50 of 65
indeed effective (% of staff trained is not an outcome oriented indicator!). It is unfortunate that
no more efforts were made to capture the changes induced by the training courses because the
anecdotal reports are generally positive.
Farmer Register
As described in Section 3.4 a comprehensive exercise was done to register farmers in 26
districts. CEOs collected information on all farmers in their camp, often as many as 5,000. The
data on paper questionnaires was inserted by the CEO in an Excel database file. The register may
result in enhanced planning of the work by the CEO- but this will depend much on the initiative
of the individual CEO. The register’s main expected benefits are expected to be in the area of the
FISP programme, whose difficulties in targeting the intended smallholders are generally known
and even mentioned in the CAADP NAIP. GRZ/MAL has been intending to implement e-
voucher and other digital schemes, which would allow for much better targeting and monitoring
of input distribution. No decision has yet been made on the most suitable methodology to apply,
but it is certain that a digital, mobile solution will be used.
At that time, the Excel database with the Farmers Register is most likely to be obsolete and a
whole new digital Farmers Register will have to be collected including ID card numbers and/or
fingers prints. Even though it is acknowledged that GRZ has delayed the implementation of the
e-voucher for various reasons, it still seems likely that the money spent on the Farmers Register
by PEP will prove to be a not a very cost-effective investment.
In the meantime, the Excel-based paper register may well have a positive effect on the
identification of the deserving FISP beneficiaries, but PEP has not put any system in place (yet)
to measure this effect.
Strengthening Regional Research Stations and Institutions
During PE2, equipment will be purchased and delivered to several research stations and
institutes: the Zambia Agricultural Research Institute (ZARI), the Seed Control and Certification
Institute (SCCI) and the Livestock Department. The rationale is described as follows: “In line
with the modern role of MAL in providing an enabling environment for private sector led
agricultural development, the ministry has an important role in providing independent analytical
services to producers, distributors and farmers for seeds, fertilisers, livestock feeds and soils.
These include germination testing, soil analysis, feeds analysis and fertiliser nutrient analysis and
these services need to be made available throughout the country.”
This statement appears rather contradicting; is the equipment needed to allow the government to
improve its role as a regulating and facilitating body? Or is it needed for the government to
provide (indirectly subsidised) services that will compete with and therefore hamper private
sector development?25
Nevertheless, it appears that in any case the equipment will be effective in allowing the
government to assume its regulatory role (e.g. by ensuring the feed or fertiliser is according to
the provided specs).
25
TA Team Leader sees no conflict as independent labs are not generally available in Zambia, so the government
needs to provide impartial services to supplement private sector ones
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 51 of 65
It seems unlikely that the provision of these services will help the institutions to gain financial
self-sufficiency as all revenues earned have to be lodged on the government account.
Payment of Personal Emoluments
PEP provided the funds for the payment of the outstanding Personal Emoluments for retired or
dismissed staff in Southern and part of Eastern Province up to 2011 (see also Section 3.5,
Activity 3.1). The assumption that the payment of these emoluments would lead to an increase in
recruitment of new staff, proved incorrect. The recruitment procedures are not related to the
payment of the final emoluments, but occur as soon as the position is registered as free (unless
there is a freeze on recruitment as is currently the case, or other budgetary restrictions apply).
The main positive effect of the emolument payments was the fact that the retirees would leave
their government houses, which they were occupying until outstanding dues were paid. This
means that the newly recruited staff could now move to the house corresponding to their
occupation. This positive effect was most prominent at the research stations, where previous
directors were blocking the flow. In other instances even vacating the house did not solve the
problem completely because no money was available to repair the, often dilapidated, house. This
is especially the case for CEOs, whose camp houses generally require significant maintenance
work prior to the arrival of a new tenant. Another (but again anecdotal!) benefit was that the
supervising MAL staff was able to work more efficiently, once they were no longer harassed by
the retirees for their payments.
4.4 Impact
Considering the variety of actions that PEP has provided to MAL, it seems likely that there will
be a small positive impact of PEP on the Overall Objective of The Agriculture sector contributes
to growth and poverty reduction in Zambia”.
It can be assumed (but not ascertained) that indeed the different training courses have imparted at
all levels the relevance of the CAADP process. Whether the poverty agenda has been fully taken
on board seems doubtful, as this was not part of the training programmes. Nevertheless, it can
also be assumed that better equipped and better trained CEOs will provide better extension
services; whether they do indeed include those living in extreme poverty seems less likely.
The third OO indicator of a reduction in FISP and FRA subsidies is curious as this would only be
a very indirect impact of the programme. Furthermore, it is acknowledged that FISP and FRA
are badly executed but these types of programmes are not always harmful (in Malawi, the EU is
a major donor in the subsidy programme).
4.5 Sustainability
On sustainability, one checks whether programme activities will continue to give results after its
phase out. This means there must be mechanisms in place to enhance its continuity.
Sustainability of PEP´s potential effects in the long terms depends on whether there are support
structures and systems to achieve full ownership by MAL and whether this ownership will
cascade to the province, district, block and levels. This also depends on whether the PEP will be
able to provide an effective contribution to the sector development, which is a factor that does
not seem to have been realised, especially due to the slow pace. Sustainability also strongly
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 52 of 65
depends on the political will. Another key consideration here is also whether MAL will fully
embrace modernisation and 'new ways of doing business'.
Institutional ownership and sustainability
At the PEP's inception there were insufficient group meetings with directors and CMT meetings.
This led to delays in decision making on priority activities for the SUPE. During PE1 and now
PE2 the situation has improved, but it is more of coordination meetings rather than championing
the change. Of paramount importance is to develop mechanisms that will support the change
circle and have it institutionalised across all levels of MAL. These mechanisms are missing more
especially at the province, district, camp and block levels, so a deep sense of ownership is still
elusive.
Financial viability for target group
The programme is expected to help in improving the sector´s financial management and improve
budget execution, with considerable benefits to the overall financial efficiency. This has not been
the case so far. PEP also targeted to improve MAL's managerial acumen which would lead to
more efficient personnel cost administration. Currently, the workforce has been trained on IFMIS
but there is no supporting equipment for operationalising this process. Hence the financial
viability of MAL is still questionable.
Capacity building and networking of partners for sustainability
Capacity building is key to enhance activities that are more related with achieving the OO of
PEP, including but not limited to research and development, food security and nutrition, fishing,
crop and livestock marketing, technology, infrastructure, improved extension services, soil
fertility management and enhancement and irrigation. Donor interventions on agriculture have
tended to focus on building capacity within public sector bodies, while, according to EU studies,
more could be done to strengthen demand for improved policy, and to create opportunities as
mentioned above for interaction between stakeholders in government and the private sector.
Strengthening and capacitating MAL should, therefore, be seen as part of a broader agricultural
sector development process.
Equipment
Large tenders for equipment are currently still outstanding and much equipment will be procured
before the end of the programme. It is imperative that MAL allocates sufficient budget for
maintenance and operation of this equipment. Considering the rather erratic MAL disbursements
in the past, this could constitute a risk that threatens the sustainable use of the hardware.
GRZ contribution
The FA specifies that the GRZ is not expected to contribute any funding. This is usually not
conducive to creation of ownership or the responsible use of funding. In the case of the Farmers’
Register, a considerable amount of funding has been consumed to collect data according to a
methodology of which is already certain that this will not be replicated. The paper questionnaires
had to be manually inserted in locally available ICT equipment. In certain cases, the CEOs felt
obliged to use their own laptops. However, this problem will be resolved soon, once the ICT
tender has been finalised and delivered.
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 53 of 65
The Seed Control and Certification Institute (SCCI)
The Seed Control and Certification Institute constitutes a good example of how a government
institution can be contributing to the creation of an enabling environment for private sector
development whilst at the same time providing a regulatory and legislative framework that will
protect the farming community from irregular practices. The Institute is international accredited
and has been creating such a conducive environment that the seed sector is thriving in Zambia,
attracting international seed companies. The institute has always been able to generate a
significant part of its own resources, collected from fees and licences, but since recently, these
revenues all have to be deposited in the General Budget.
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 54 of 65
5. Recommendations
The specific objective of the MTE according to the ToR was to “provide recommendations for
any required change/modification to project scope (including objectives, management
arrangements, financing, etc.) in order to support effective implementation and the attainment of
the expected results by the end of the project. It should be noted that the MTE came at a rather
late stage of programme implementation, with the programme well in the implementation of its
second and last PE and with less than one year remaining.
The global objective of the assignment also pointed to the follow-up programme, as the MTE
was “to ensure that this informs the design of the forthcoming "Enhanced Capacity of the
Ministry of Agriculture for Better Service Delivery to Farmers" project.”
5.1 Recommendations for the remaining lifetime of PEP
a) It should be recognised that some of the expected results are composed of two separate
results. It is recommended that a separate LFM is drafted for each separate result area,
including an implementation path with key progress indicators. This is expected to assist the
programme in identifying its direction and will also allow better assessment of the
effectiveness of the separate activities.
b) It is further recommended that the CMT of PEP articulates precisely what the desired change
is that the programme aims to entail. It is probably enhanced service delivery (which figures
currently in the results) and the programme should identify suitable indicators to assess how
and whether the activities performed under each component do lead to enhanced service
delivery and how this will contribute to improved productivity and enhanced food and
nutrition security for smallholder producers.
c) It is further recommend the entire CMT team is trained in Project Cycle Management (PCM).
This is the recommended approach for programmes implemented with EU funding.
Furthermore, this MTE has concluded that a better understanding of PCM principles would
have benefitted both implementation as well as capturing effectiveness and impact of the
activities.
d) It is further recommended that the concerned research stations and institutions demonstrate
what benefits will be received from procured items. GRZ/MAL should commit to allowing a
certain degree of independence, allowing the institutions a certain degree of autonomy.
e) It is further recommended that the CMT team demonstrates a more pro-active attitude and
takes full responsibility for the actions that concern their own particular area. The Technical
Assistance team should be allowed to concentrate more on the provision of technical
assistance and less on actual implementation.
f) Several proposed activities and outputs are still in the early stages of implementation (M&E
system, Functional Review, Enhanced planning, ICT operational). It is recommended that
these are critically studied and that an assessment is made of the feasibility that these will
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 55 of 65
indeed be accomplished before the end of the programme. Possible adjustments or
accompanying measures will have to be defined.
5.2 Recommendations for the 11th EDF follow-up programme
a) The MTE has acknowledged a good number of achievements of PEP, and more are still
expected in the remaining year. It is recommended that the follow-up programme builds on
these achievements where possible and to learn lessons from them where appropriate, for
example:
Build on the leadership, management and supervisory training, but ensure the managers
at different levels develop a common understanding of the overall change process;
Build on the experience of extension harmonisation;
Make use of farmer registers to better target interventions;
Assist in the implementation of the Strategic Plan;
Continue training of staff, but with clear reference to the programme purpose and
objective, and with systems in place to measure effectiveness;
Continue with the rolling out of ICT systems at all levels of the ministry;
Apply the policy and guidelines that were developed in HIV/AIDS and gender, to
genuinely mainstream these cross-cutting issues in the programme;
Support the implementation of the MAL M&E manual that PEP made possible;
b) It is further recommended that the new programme is based on a solid intervention logic,
which is finalised in the earliest stages of implementation, with all major stakeholders
involved. The logical framework is to be used as a tool for M&E of the programme, ensuring
a consistent, well-targeted programme.
c) It is further recommend that SMART indicators are developed as part of the logical
framework, with baseline and target values.
d) It is further recommended that the new programme adopts a strong focus on poverty, actively
targeting poor rural households and measuring impact at household level.
e) It is further recommended that the new programme commits more attention to
implementation at province and district level (inlc. block and camp). For maximum impact it
is recommended to concentrate certain activities in one or two provinces. Lessons from these
provinces can be applied throughout the country.
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 56 of 65
Annexes
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 57 of 65
Annex 1 Terms of Reference
To be inserted
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 58 of 65
Annex 2 Itinerary: people and institutions consulted
Date Activity/Institution Name and designation
Monday 18
August
Briefing at EUD Gilles Hervio, Head of Delegation
Aad Biesebroek, Head of Cooperation
Kirsi Pekuri, Head of Section
James McNulty, Result Adviser
Briefing at NAO Temwani Chihana, EDF Coordinator
Timothy Nyirenda, NAO Programme Officer
Eleonor Phiri Ngwira, Accounting Officer
Kondwani Gondwe, MAL, PPD
Jim Parker, TA Team Leader
Kirsi Pekuri, EUD, Head of Section
James McNulty, EUD, Result Adviser
Briefing at PSU Jim Parker, TA Team Leader
James McNulty, EUD, Result Adviser
Tuesday 19
August
Meeting at EUD,
Finance and
Contracts
Christian Creusen, Head of Section F&C
Nathalie Dekeyser, Audit Task Manager
Meeting at EUD,
Economics, Private
Sector and Rural
Development
Kirsi Pekuri, Head of Section
Meeting at PSU Jim Parker, TA Team Leader
Bursch Nketani, Human Resource Adviser
Meeting at MAL Julius Joseph Shawa, Permanent Secretary
Emma Malawo, Director PPD
Derrick Sikombe, Chief Agricultural Economist
Kondwani Gondwe, PPD
Timothy Nyirenda, NAO Programme Officer
Jim Parker, TA Team Leader
Kirsi Pekuri, EUD, Head of Section
James McNulty, EUD, Result Adviser
Wednesday
20 August
Meetings at PSU Jim Parker, TA Team Leader
Bursch Nketani, Human Resource Adviser
Kanyifa Mvula, Programme Accountant
Esther Sikazwe, Administration Officer
Meeting with MAL Julius Joseph Shawa, Permanent Secretary
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 59 of 65
Date Activity/Institution Name and designation
– CMT Emma Malawo, Director PPD
Moses Mwale, Director ZARI
Catherine Mungoma, Director SCCI
Patrick Ngalande, Director Fisheries
Benson Mwenya, Director Livestock
Joseph Mubanga, Director Veterinary Services
Jane Phiri, Ag. Director HRA
Shadreck Mungalaba, Ag. Director Coops
Kezia Katyamba, Ag. Director ABM
Charles Sondashi, Deputy Director DOA
Derrick Sikombe, Chief Agricultural Economist
Kondwani Gondwe, PPD
Chimbala Yo Yo, PPD
Timothy Nyirenda, NAO Programme Officer
Thursday 21
August
Meetings at MAL Emma Malawo, Director PPD
Peter Kunda, PPD
Cosmore Mwaanga, PPD
Jane Phiri, Ag. Director HRA
Lois Mulube, HRA
Remmy Chibiya, HRA
Christine C. Yamba Yamba, Deputy Director
Livestock
Charles Sondashi, Deputy Director, DOA
Joseph Mubanga, Director Veterinary Services
Meeting at EUD Kirsi Pekuri, Head of Section
James McNulty, Result Adviser
Andrew Zulu, Administration
Preparation of draft
inception report
Friday 22
August
Meetings at MAL Shadreck Mungalaba, Ag. Director Coops
Kezia Katyamba, Ag. Director ABM
Derrick Sikombe, Chief Agricultural Economist
Meeting at SCCI Catherine Mungoma, Director SCCI
Meeting at ZARI Monde Siyandwa Zulu, Deputy Director ZARI
Discussion of draft
inception report
Aad Biesebroek, Head of Cooperation
Kirsi Pekuri, Head of Section
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 60 of 65
Date Activity/Institution Name and designation
James McNulty, Result Adviser
Meeting with
RESCAP
Masayoshi Ono, Chief Adviser
Hirohiko Asada, Coordinator Training Management
Sunday 24
August
Travel to Choma,
Southern Province
Monday 25
August
Meetings at
Provincial
Agricultural Office
Adreen Nansungwe, PACO
Charles Masoka, PVO
Charity Chabaya, Executive Officer
Brian Musunda, Ag. HRMO
Zandonda Tembo, Ag. Senior Marketing Officer
Jane Nambela, Seed Analyst
George Njobvu, Controller NAIS
Max Choombe, DACO Kalomo
Goliath Chooye, Ag. DACO Choma
Meetings at Choma
District Agricultural
Office
Goliath Chooye, Ag. DACO Choma
Banda Fundi, Senior Agriculture Officer
Gwen Miyoba, Agr. Assistant Siakacheka Camp
Brian Musunda, Ag. HRMO
Seed Co Manager local branch with SCCI officer
Visit maize purchase
point
2 FRA purchase officer, with DACO marketing officer
Members Choma Farmers Coop
Tuesday 26
August
Meeting at Kalomo
District Agricultural
Office
Max Choombe, DACO Kaloma
Moses Mumba, Ag. SAO
Aaron Phiri, Mechanisation Officer
Visit Malindi Camp Roy Chinda, Agr. Assistant Malindi Camp
Ms Sakrina, Ms Jeanette, women farmers
Beauty Hamalambo, woman farmer
Mr & Mrs Boyd Mweemba, farmers
Visit Choonga Camp Mildred Meandasioni, Agr. Assistant Choonga Camp
Mr & Mrs Bernhard Hakalima, farmers
Visit Kaloma
Farmers Training
Centre
Meeting at Zambia
Farmers Union
Hamish Ross, Vice Chairman
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 61 of 65
Date Activity/Institution Name and designation
Chonga
Wednesday
27 August
Visit Zambia
College of
Agriculture, Monze
Likando Muyatwa, HoD Agro economics and RD
Anna Mwansa, Training Officer
Emily Mwananyau, Accountant
Visit Zambia
Institute of Animal
Health, Mazabuka
Travel to Lusaka
Thursday 28
August
Meetings at MAL Kondwani Gondwe, PPD
Nancy Chela, Chief Nutritionist
Karen Chenda Mukuka, Chief Food and Nutrition
Officer (MAL)
Andrew Mumo, Project Officer, Dept of Agriculture
Meeting at PSU Jim Parker, TA Team Leader
Meeting at Embassy
of Finland
Nachili Kaira, Sector Adviser Agriculture and RD
Friday 29
August
Attend M&E Donor
Coordination
Meeting
Jim Parker, TA Team Leader
Derrick Sikombe, Chief Agricultural Economist
No more members showed up so the meeting did not
take place
Meetings at MAL Charles Sondashi, Deputy Director, DOA
Jane Phiri, Ag. Director HRA
Meeting with TA TL Jim Parker, TA Team Leader
Meeting with
RESCAP
Masayoshi Ono, Chief Adviser
Hirohiko Asada, Coordinator Training Management
Goichi Sasaki, Agricultural Extension Adviser
Saturday 30
August
Drafting MTE report
Sunday 31
August
Drafting MTE report
Monday 1
September
Meeting at VP’s
office, MDD
Abdu Sakala, Director Performance Systems
Tuesday 2
September
Drafting MTR &
presentation
Wednesday 3 Recap meeting at Aad Biesebroek, Head of Cooperation
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 62 of 65
Date Activity/Institution Name and designation
September EUD Kirsi Pekuri, Head of Section
Marion Michaud, Agr. and Rural Development
Advisor
James McNulty, Result Adviser
Nicholas Chikwenya, Deputy Director PPD
Jim Parker, TA Team Leader
Bursch Nketani, Human Resource Adviser
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 63 of 65
Annex 3 Documentation consulted
Revised Sixth National Development Plan (R-SNDP) 2013-2016
National Agriculture Policy (NAP) 2004 – 2015 currently under review
Comprehensive African Agriculture Development Programme - Zambia Compact
National Agriculture Investment Plan (NAIP) 2014-2018
Strategic Plan of the Ministry of Agriculture and Livestock, 2013-2016
Project Identification Fiche “Enhanced Capacity of the Ministry of Agriculture for Better
Service Delivery to Farmers Project”, April 2014 and annexes and conclusions of QSG of
22 May 2014
Financing Agreement (FED/2010/022-057), Start Up Programme Estimate and
Programme Estimates no. 1 and 2 and related progress and financial reports
Project deliverables including thematic studies, training materials, policy briefs, etc.
Results-Oriented Monitoring Reports and Background Conclusions Sheets, 2012 and
2013
“Participatory Review of the Ministry of Agriculture and Co-operatives Performance
[...]", including an "Assessment of Current MACO Capacity, Capacity Development
Needs and Challenges" and a "Strategy and Roadmap for a Performance Enhancement
Programme in MACO/GRZ", Cardno Agrisystems Ltd, July 2009
General Operational Guidelines for Agricultural Extension Service Providers
MAL Performance Enhancement Programme; Needs Review, January 2012
Report on the Assessment of small-scale farmers’ perception on the farmer input support
programme (FISP) reforms, MAL Policy and Planning Department, May 2014
How can the Zambian Government Improve the targeting of the farmer input support
programme? IAPRI Policy Brief, nr 59, Rhoda Mofya-Mukuka et al
Can the FISP more actively achieve food production and poverty reduction goals?
William Burke et all, Food Security Research Project, Zambia, March 2012
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 64 of 65
Annex 4 Key evaluation questions
Below are presented key questions that were used to guide the evaluation process.
Relevance
1.1 What is the present level of relevance of the PEP?
1.2 As presently designed, is the intervention logic still valid?
1.3 Is the current design sufficiently supported by all stakeholders?
1.4 Is the current design sufficiently taking cross-cutting issues into account?
Efficiency of implementation
2.1 How well is the availability/usage of means/inputs managed?
2.2 How well is the implementation of activities managed?
2.3 How well are outputs achieved?
2.4 How well is the Partner Contribution/Involvement working?
2.5 What M&E mechanisms are in place? Are M&E results used to improve effective
implementation? Are PCM practices adhered to?
Effectiveness of the programme
3.1 How well is the project achieving its planned results?
3.2 As presently implemented what is the likelihood of the PP to be achieved?
Impact
4.1 What are the direct impact prospects of the project at Overall Objectives level?
4.2 To what extent does/will the project have any indirect positive and/or negative impacts?
Potential Sustainability
5.1 Will MAL be able at the end of the programme to sustain the systems and mechanisms of the
programme from its own resources, or are there other sources of funding available?
5.2 What is the level of ownership of the project by target groups and will it continue after the
end of external support?
5.3 What is the level of policy support provided and the degree of interaction between project
and policy level?
5.4 How well is the project contributing to institutional and management capacity?
Cross-cutting Issues
6.1 Have practical and strategic gender interests been adequately considered in the project
strategy?
6.2 Is the project respecting environmental needs?
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 65 of 65
Annex 5 Reference model of Change Management
To begin any successful change process, first there is need to start by understanding why the
change must take place , which involves breaking down the existing status quo before you can
build up a new way of operating . "Motivation for change must be generated before change can
occur." Develop a compelling message showing why the existing way of doing things cannot
continue. Change by its very nature require a shift into some better modus operandi.
This is easiest to frame when one can point to insufficient service delivery and low capacity
levels that influence the public image. To prepare the organisation successfully, one needs to
start at its core -there is need to challenge the beliefs, values, attitudes, and behaviours that
currently define it (culture). When you start cutting down the "way things are done", you put
everyone and everything off balance. Strong reactions in people may be evoked, and that's
exactly what needs to be done for change to happen.
By looking at change as process with distinct stages, one can prepare for what is coming and
make a plan to manage the transition. Suggested below is a ten steps change model for
effectively leading change:
Change Management Model
(1) Create a Need /Urgency for change
(9) Anchor the change
(8) Build on the change
(7)Create short wins
(6) Remove obstacles/ Enhance change enablers
(4)Communicate the vision/targets
(3) Develop a vision& short term targets
(2)Build a Team/ Coalition
(5) Activities that support the vision / targets
Changing /Shifting / Moving from state to another
M&E M&E
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 66 of 65
Step 1: Create Urgency for change
For change to happen, it helps if the whole organisation really wants it. Develop a sense of
urgency around the need for change. This may help you spark the initial motivation to get things
moving. Open an honest, dynamic and convincing dialogue about what's happening and what
your customers expect so as to get people thinking and talking about change.
Step 2: Form a Powerful Team/Coalition
Through strong leadership and visible support from key people within your organisation,
convince people that change is necessary. Managing change isn't enough – it needs to be led. To
lead change, bring together a coalition, or team, of influential people whose power comes from a
variety of sources, including job title, status, expertise, and political importance (different
departments and stakeholders).
Once formed, your "change coalition" needs to work as a team, continuing to build urgency and
momentum around the need for change. They champion change, they are change agents.
Step 3: Develop a Vision for Change
When you first start thinking about change, there will probably be many great ideas and solutions
floating around. Link these concepts to an overall vision that people can grasp easily and
remember. A clear vision can help everyone understand why you're asking them to do
something. When people see for themselves what you're trying to achieve, then the directives
(short term targets) they're given tend to make more sense.
Step 4: Communicate the Vision
What you do with your vision after you create it will determine your success. You need
to communicate it frequently and powerfully, and embed it within everything that you do. Don't
just call special meetings to communicate your vision. Instead, talk about it every chance you
get. Use the vision daily to make decisions and solve problems. When you keep it fresh on
everyone's minds, they'll remember it and respond to it. "Walk the talk." Demonstrate the kind of
behaviour that you want from others. Lead by example.
Step 5: Actvities that support the Vision
Based on the result areas, carry out activities that are in line with the vision and the short term
targets. Reposition and channel all the actions, energy and efforts towards achieving the vision.
Step 6: Remove Obstacles
Hopefully, your staff wants to get busy and achieve the benefits that you've been promoting. But
is anyone resisting the change? And are there processes or structures that are getting in its way?
Put in place the structure for change, and continually check for barriers to it. Removing obstacles
Mid-Term Evaluation of the " Performance Enhancement Programme" (10th EDF) & Formulation of the "Enhanced Capacity of the MAL for Better Service Delivery to Farmers Project" (11th EDF)
Final Mid-Term Evaluation Report – September 2014 Page 67 of 65
can empower the people you need to execute your vision, and it can help the change move
forward.
Step 7: Create Short-Term Wins
Nothing motivates more than success. Give your organisation a taste of victory early in the
change process. Within a short time frame (4months or a year, depending on the type of change),
you'll want to have some "quick wins” that your staff can see. Without this, critics and negative
thinkers might hurt your progress.
Create short-term targets – not just one long-term goal. You want each smaller target to be
achievable, with little room for failure. Your change team may have to work very hard to come
up with these targets, but each "win" that you produce can further motivate the entire staff.
Step 8: Build on the Change
Real change runs deep. Quick wins are only the beginning of what needs to be done to achieve
long-term change. To reach that 10th success, you need to keep looking for improvements. Each
success provides an opportunity to build on what went right and identify what you can improve.
Step 9: Anchor the Changes in Organisational Corporate Culture
Finally, to make any change stick, it should become part of the core of your organisation. Your
corporate culture often determines what gets done, so the values behind your vision must show in
day-to-day work.
Make continuous efforts to ensure that the change is seen in every aspect of your organisation.
This will help give that change a solid place in your organisation's culture.
It's also important that your company's leaders continue to support the change. This includes
existing staff and new leaders who are brought in. If you lose the support of these people, you
might end up back where you started.
Step 10: Monitoring and Evaluation
For change to happen effectively, there is need to check each of the nine steps in terms of what is
planned for and what happens practically. Due adjustments are key to ensure effectiveness of
each step. At step 5, clear indicators will be used to assess effectiveness and impact.