project review of the dairy association of zambia (daz)’s

76
Sida Decentralised Evaluation Project Review of the Dairy Association of Zambia (DAZ)’s Digital Information Management System (DIMS) project Final Report NIRAS Sweden AB 2020:9

Upload: others

Post on 23-May-2022

5 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Project Review of the Dairy Association of Zambia (DAZ)’s

Sida Decentralised Evaluation

Project Review of the Dairy Association of Zambia (DAZ)’s Digital Information Management System (DIMS) project

Final Report

NIRAS Sweden AB

2020:9

Page 2: Project Review of the Dairy Association of Zambia (DAZ)’s

Project Review of the Dairy Association of Zambia (DAZ)’s Digital Information

Management System (DIMS) project

Final ReportApril 2020

Åke NilssonPia Chuzu

Sida Decentralised Evaluation 2020:9Sida

Page 3: Project Review of the Dairy Association of Zambia (DAZ)’s

Authors: Åke Nilsson and Pia Chuzu

The views and interpretations expressed in this report are the authors’ and do not necessarily reflect those of the Swedish International Development Cooperation Agency, Sida.

Sida Decentralised Evaluation 2020:9

Commissioned by Sida, Embassy of Sweden in Zambia

Copyright: Sida and the authors

Date of final report: 2020-04-09

Published by Nordic Morning 2020

Art. no. Sida62303en

urn:nbn:se:sida-62303en

This publication can be downloaded from: http://www.sida.se/publications

SWEDISH INTERNATIONAL DEVELOPMENT COOPERATION AGENCY

Address: SE-105 25 Stockholm, Sweden. Office: Valhallavägen 199, StockholmTelephone: +46 (0)8-698 50 00. Telefax: +46 (0)8-20 88 64E-mail: [email protected]. Homepage: http://www.sida.se

Page 4: Project Review of the Dairy Association of Zambia (DAZ)’s

Table of contents ConContent CContent contents

i

Table of contents ...................................................................................................................... i

Abbreviations and Acronyms ................................................................................................. ii

Preface ..................................................................................................................................... iii

Executive Summary ................................................................................................................ iv

1 Introduction ......................................................................................................................... 1

2 The Evaluated Intervention ................................................................................................ 2

2.1 Background .................................................................................................................. 2

2.2 Evaluation object .......................................................................................................... 3

2.3 Objectives and methodology ........................................................................................ 5

3 Findings ............................................................................................................................... 8

3.1 Relevance .................................................................................................................... 8

3.2 Effectiveness .............................................................................................................. 12

3.3 Efficiency .................................................................................................................... 15

3.4 Sustainability .............................................................................................................. 16

3.5 Crosscutting issues .................................................................................................... 17

4 Evaluative Conclusions.................................................................................................... 23

5 Recommendations ............................................................................................................ 24

5.1 Recommendations to DAZ.......................................................................................... 24

5.2 Recommendations to Embassy of Sweden ................................................................ 25

Annex 1 – Terms of Reference.............................................................................................. 27

Annex 2 – Evaluation Matrix ................................................................................................. 36

Annex 3 – Documents studied .............................................................................................. 41

Annex 4 – Persons interviewed ............................................................................................ 42

Annex 5 – Beneficiary questionnaire ................................................................................... 46

Annex 6 – Results framework with reported results ........................................................... 48

Annex 7 Comments on selected indicators in DAZ DIMS result framework ..................... 58

Annex 8 Monthly Staff Costs ................................................................................................ 61

Page 5: Project Review of the Dairy Association of Zambia (DAZ)’s

Abbreviations and Acronyms AAcronyms

ii

7NDP 7th National Development Plan, 2017-2021, Zambia

DAZ Dairy Association of Zambia

DFAP Digital Farmer Aggregation Platform

DIMS Digital Information Management System

FFA Force Field Analysis

FISP Farmer Input Support Programme

FRAS Fair Resource Allocation System

GDP Gross Domestic Product

HRBA Human Rights Based Approach

MCC Milk Collection Centres

M&E Monitoring and Evaluation

MPS Mobile Payment Solutions

SACAU Southern African Confederation of Agricultural Unions

Sida Swedish International Development Cooperation Agency

SEK Swedish Krona

ToR Terms of Reference

ZDTP Zambia Dairy Transformation Programme

ZICTA Zambia Information and Communication Technology Agency

ZK Zambia Kwacha (currency)

ZNFU Zambia National Farmers’ Union

Page 6: Project Review of the Dairy Association of Zambia (DAZ)’s

Preface

iii

The Embassy of Sweden, Lusaka, has contracted NIRAS to conduct a project review

of the Dairy Association of Zambia (DAZ)’s Digital Information Management System

(DIMS) project in Zambia. The purpose of the review has been to provide conclusions

and recommendations to help the Embassy in assessing project progress and providing

recommendations for eventual continued funding, and to provide input to DAZ on how

to adjust and improve project implementation and to provide recommendations for their

future programming.

The report has been prepared by an evaluation team with the following

members:

• Åke Nilsson (team leader and environment specialist)

• Pia Chuzu (evaluator and agriculture specialist)

Christina Paabøl Thomsen has managed the review at NIRAS and Ted Kliest has

provided quality assurance.

NIRAS and the review team would like to thank stakeholders at DAZ and the Embassy

of Sweden for the time and support they have provided to the evaluation. The findings

and recommendation of the report are those of NIRAS and the evaluation team.

Page 7: Project Review of the Dairy Association of Zambia (DAZ)’s

Executive Summary

SummSummary

iv

This report presents the results of an independent project review of the Digital

Information Management System (DIMS) Project, implemented by the Dairy

Association of Zambia (DAZ), hereafter referred to as ‘the project’. The project was

granted SEK 16 million in funding from Sida to implement pilot activities for 2 years

from June 2018 to May 2020.

The overall aim of the project is to contribute to a viable and sustainable dairy industry

that positively contributes to the socio–economic development of Zambia through

contribution to increased national income and employment.

The specific objectives of the project are to:

1. Digitise transactions of Milk Collection Centres (MCCs).

2. Reduce the cost of doing business for targeted small-scale dairy farmers.

3. Improve access to information, inputs and equipment by small-scale dairy

farmers.

4. Improve financial inclusion for targeted small-scale dairy farmers.

The project is a pilot project, covering 32 MCCs out of a total of 74 MCCs in the

country.

The purpose of the review has been to provide conclusions and recommendations

intended to help the Embassy of Sweden in Lusaka to assess project progress and

providing recommendations as a basis for their decision on whether to continue funding

the project. The review also provides input to DAZ on how to adjust and improve

project implementation by identifying areas of possible improvement, and give

recommendations for future programming.

A multi-pronged approach was used to collect and triangulate data from primary and

secondary sources. Written sources were reviewed to obtain an overview of the project

and to assess progress. This was followed by key informant interviews with the

Swedish Embassy, DAZ staff and other stakeholders in Lusaka. Field visits were made

to eight MCCs in four different provinces, during which focus group discussions and a

questionnaire survey were carried out. Two workshops were carried out with DAZ

staff, one on performance related to the project result framework and one on force-field

analysis.

Page 8: Project Review of the Dairy Association of Zambia (DAZ)’s

E X E C U T I V E S U M M A R Y

v

Based on the findings of the evaluation, the following evaluative conclusions are

made:

- The project is highly relevant. It fits well with relevant strategic and policy

documents and should, if successful, provide substantial benefits to its stakeholders

and intended beneficiaries.

- Judging from the newly reported current indicator values, the project has improved

its effectiveness during the later stage of implementation. However, the current size

of the pilot is too large to allow for optimal effectiveness, and the project would

benefit from being scaled down and producing more tangible results to DIMS users

rather than scaled up.

- The project and its results can potentially be sustainable since there is willingness

among farmers to pay for the system as long as the costs are clarified and its value

clearly demonstrated. However, this requires that the project is able to show more

benefits emanating to the beneficiaries from the actual DIMS than what is currently

the case.

- It is concluded that while there have been some outputs that should benefit

environmental sustainability, including demonstration and training related to

pasture management and fodder, no strategic environmental assessment has been

carried out.

- While the structure is top-heavy with many staff at the secretariat, the presence of

DAZ staff at the MCC level, notably the field facilitators, has been reported to be

relatively limited. The field facilitation will become even more important in the

future as the DIMS services develops and this may mean that the distribution of

staff capacity between secretariat and field staff needs to be reassessed. This is the

case also with regard to the current professional profiles and competence in

emerging themes such as climate change and HRBA.

- Action to adapt to climate change is a priority theme for DAZ to address and this

should be done through cooperation with partners. The evaluation team concludes

that SNV would be a suitable partner in this regard. Another potential cooperation

partner would be ZDTP. To the extent that in-house competence in such areas is

deemed necessary and acquired, this needs to be matched with a corresponding

reduction in staff with other profiles.

- While the project will serve to increase benefits for people living in poverty

following from its provision of services to smallholder dairy farmers, there is no

strategy for how to reach the most marginalised farmers or how to make the ser-

vices useful for illiterate persons.

- Although a gender policy has been prepared, this has not yet been ratified or made

operational.

- Youth and women cannot fully participate in mainstream dairy farming activities

because few own dairy cattle, most youth are not interested in dairy farming and

both women and youth are interested in other activities along the dairy value chain

but lack resources for investment.

Page 9: Project Review of the Dairy Association of Zambia (DAZ)’s

E X E C U T I V E S U M M A R Y

vi

The following recommendations are made to DAZ:

Policy

- The DAZ Board should ratify the gender policy to enable DAZ to implement the

activities suggested in the policy. An action plan for implementation should

prioritize activities based on what can feasibly be attained within the timeframe of

the project and also considering previous recommendations from the gender audit

of 2019.

Strategy

- Follow up the results of the forcefield analysis workshop with a strategic planning

activity aiming at improving project implementation effectiveness.

- Speed up implementation related to finalising profiling, developing the system to

meet the more advanced needs of the beneficiaries, and providing inputs under the

Memoranda of Understanding in order not to lose the interest of the beneficiaries.

- In order to achieve the previous point, scale down and intensify rather than scale

up and out to the extent this is possible keeping in mind that the system service

provider needs to have a certain volume of business.

- The DIMS project needs to produce and publicize an exit strategy and ensure that

all participating MCCs and their members are made aware of their responsibilities

to sustain the benefits after project end. Further, all costs to be borne need to be

clarified. A phased approach for takeover of the DIMS platform and associated

costs is advised.

- Carry out a strategic environmental assessment and prepare an action plan for its

work with environment and climate change mitigation, adaptation and resilience,

including how to cooperate with partners. Indicators for implementation of such a

plan should be included in the project result framework, and the implementation of

the plan should have its own budget line in the project budget.

- The project should prepare a strategic action plan for how to reach the more

marginalised farmers. This could be done by applying an Ex Ante Poverty Impact

Assessment (PIA) methodology1, through which the project can target people living

in acute poverty specifically and more directly, as a complement to the regular

activities addressing rural poverty in a more general sense. Indicators for

implementation of such a plan would need to be included in the project result

framework, and the implementation of the plan should be covered by a specific

budget line in the project budget.

- To increase the participation of youth and women, a separate sub-component focus-

ing on youth and women empowerment to develop small businesses should be

considered. Activities under this component should include capacity building for

these groups to identify and develop business plans for projects of interest along the

value chain, training on business management and leadership, and provision of

start-up capital or linkage to special funds for women and youth to obtain start- up

capital.

1 http://www.value-chains.org/dyn/bds/bds2search.details2?p_phase_id=645&p_lang=en&p_phase_type_id=2

Page 10: Project Review of the Dairy Association of Zambia (DAZ)’s

E X E C U T I V E S U M M A R Y

vii

Technical and planning

- The measures and recommendations need to be prioritized and focus should be on a

few areas that can be feasibly addressed as not all measures can be implemented

during the life of the project.

- Conduct a needs assessment at each MCC and prioritize what is to be delivered in

the remaining project period and a continued phase.

- Engage constantly with the beneficiaries. An example of such interaction could be

facilitated quarterly self-evaluation/lesson sharing events at MCC level.

- The project needs to engage an IT specialist on the project team who is technically

knowledgeable about the offerings from the service providers in order for DIMS to

interact with the service provider on a more informed footing.

- Intensify training, based on training needs assessment for each MCC specifically,

combined with relevant follow-up activities in the field.

- To address the limiting factor of illiteracy, the inclusion of audio modules of

extension that illiterate farmers can access at cooperative level should be

considered. With the help of field facilitators some lead farmers should be

identified who can help organize an orderly way to access this audio information.

- Enhance the presence of extension workers at the MCCs by way of additional

recruitment in some MCCs.

- Initiate discussion with SNV on content of cooperation related to their project

INCREASE. Discussions could also be initiated with the ZDTP to develop potential

cooperation.

The following recommendations are made to the Embassy of Sweden:

- Answering directly to the purpose of the review, it is recommended that Sida

continue supporting a continued implementation of the project, in line with the

above recommendations to DAZ.

- The remaining part of the current implementation period should be used, in addition

to implementation as recommended above, for preparation of a new three-year

project phase.

- Sida should facilitate that DAZ-DIMS can enter into a close cooperation with the

new SNV project on environment and climate-change mitigation, adaptation and

resilience, financed by Sida. Budgets that could cover such activities are already

available in both projects and no additional project or sub-project would be

required.

Page 11: Project Review of the Dairy Association of Zambia (DAZ)’s

1 Introduction Introduction

1

The Swedish International Development Cooperation Agency (Sida) has contracted

NIRAS to conduct an independent project review of the Dairy Association of Zambia

(DAZ) Digital Information Management System (DIMS) Project, hereafter referred to

as ‘the project’. The Terms of Reference for the review are provided in Annex 1.

The evaluation has been carried out by a team of two consultants, who organised two

missions, one inception mission to Lusaka during four days in November 2019, and

one two-week mission to Lusaka and four provinces in Zambia in January 2020.

Page 12: Project Review of the Dairy Association of Zambia (DAZ)’s

2 The Evaluated Intervention

2

2.1 BACKGROUND

The Swedish International Development Cooperation Agency (Sida) has contracted

NIRAS to conduct an independent project review of the Dairy Association of Zambia

(DAZ) Digital Information Management System (DIMS) Project, hereafter referred to

as ‘the project’. The project was granted SEK 16 million in funding from Sida to

implement pilot activities for 2 years from June 2018 to May 2020.

One of Zambia’s development goals is to diversify its economy from mining to

agriculture and other sectors. The agriculture sector in Zambia has traditionally focused

on crop production. With high urbanization and increased demand for livestock

products including milk and other dairy products within Zambia and in the region, dairy

production provides an opportunity for diversification within the sector. In 2011, it was

estimated that around 310,000 households owned 3 million cattle. Small-scale farmers

owned 80% of this population, with nine cattle being the average number owned.

Increasing productivity and raising incomes in the smallholder dairy sector is a way to

target rural poverty alleviation. However, productivity in the smallholder category is

low for several reasons, including lack of information and access to commercial

markets resulting in application of lower quality inputs and attendant high disease rates

among the flock. For example, milk yield per cow per day was estimated at two litres

for traditional small scale farmers compared with 17-23 litres for commercial farmers.

Zambia has the lowest milk yield and the highest price for raw milk in the region (World

Bank, 2011)1.

Although the potential to expand the dairy industry is high given Zambia’s large

grazing lands, other factors need to be addressed including disease prevention and

control, availability and access to quality inputs, better access to markets, easier access

to affordable credit, better access to rural road network and electric power

infrastructure. With respect to agriculture being the fourth largest contributor to GDP

and the largest contributor to employment especially in the rural areas, dairy production

provides an opportunity for diversification and reducing poverty within the sector.

Dairy farming is rapidly increasing in Zambia, with an estimated 1,800 and 3,500

smallholder dairy producers engaged in 2002 and 2012, respectively. In 2013, 2,330

farmers were actively supplying milk to milk collection centres (MCCs) in the country.

Between 1998 and 2013, annual milk supply from MCCs increased from 435,000 to 10

million litres, representing an annual growth rate of 23%. The total annual milk

1 World Bank, 2011: What would it take for Zambia’s beef and dairy industries to achieve their potential?

Page 13: Project Review of the Dairy Association of Zambia (DAZ)’s

2 T H E E V A L U A T E D I N T E R V E N T I O N

3

production was estimated at 200-250 million litres with annual per capita consumption

at 15-19 litres, far below the less developed country average of 75 litres (FAO, 2013)2.

The increase in milk production in Zambia is consistent with the global trend and that

for the Africa region. World-wide milk production was forecasted to increase from an

estimated 843 million tonnes in 2018 to 859 million tonnes in 2019. In Africa,

production would increase from 47.7 million tonnes in 2018 to 48.0 million tonnes in

2019. Warm and drought-prone weather which is noted to pose a threat to the rising

trend may be part of the reason for the limited increase in production in Africa (FAO,

2019)3.

Productivity is low in the Zambian smallholder dairy sector. Smallholders produce an

average of 1-3 litres compared with 8-16 litres and 20-25 litres of milk per cow per day

for emergent and commercial producers respectively. With high urbanization and

increased demand for livestock products, including milk and other dairy products, there

is opportunity for smallholder dairy farmers to increase their incomes through increased

productivity.

Although the potential to expand the dairy industry is high given Zambia’s large

grazing lands, key factors need to be addressed including disease prevention and

control, availability and access to quality inputs, better access to markets, easier access

to affordable credit, better access to rural road network and electric power

infrastructure, and better and timely access to information and extension services,

among others.

2.2 EVALUATION OBJECT The project, which is specified as a pilot project covering 32 MCCs out of a total of 74

MCCs in the country, builds on a previously implemented Digital Farmer Aggregation

Platform (DFAP). The DFAP created a digital platform for aggregation of farmers’

input purchases and produce sales, which enabled bulk procurement of services and

bulk marketing of milk in five MCCs. It was designed to be funded by the Bill and

Melinda Gates Foundation through the Southern African Confederation of Agricultural

Unions (SACAU) and the Zambia National Farmers’ Union (ZNFU). The DIMS

project, which builds upon the successes of the completed DFAP project, aims at

introducing and further developing a digital platform to facilitate input and output

aggregation, and transaction and information management.

The overall aim of the project is to contribute to a viable and sustainable dairy industry

that positively contributes to the socio–economic development of Zambia through

2 FAO, 2013: Food Outlook, November 2013. Rome, FAO.

3 FAO, 2019: FAO. Food Outlook. 2019, Biannual Report on Global Food Markets, Rome, Italy.

Page 14: Project Review of the Dairy Association of Zambia (DAZ)’s

2 T H E E V A L U A T E D I N T E R V E N T I O N

4

contribution to increased national income and employment. The theory of change for

the project is shown in Figure 1.

Figure 1. DAZ DIMS Theory of Change

Source: DIMS Semi-Annual Report 2019 Final 10022020

The specific objectives of the project are to:

1. Digitise MCC transactions;

2. Reduce the cost of doing business for targeted small-scale dairy farmers;

3. Improve access to information, inputs and equipment by small-scale dairy

farmers;

4. Improve financial inclusion for targeted small-scale dairy farmers.

Project activities include needs assessment; training of smallholder farmers on access

to and use of DIMS and in savings and financial literacy; establishment, testing and

piloting of a digital information management system; establishment of a digitised data

bank; promotion of dairy payment solutions such as digital payment systems and

insurance products; the design and operation of a web-based M&E system; and

development of digital dairy extension modules, among others.

Women and youth are targeted to benefit from project interventions through reduced

cost of dairy inputs and improved dairy produce prices, both of which are expected to

Page 15: Project Review of the Dairy Association of Zambia (DAZ)’s

2 T H E E V A L U A T E D I N T E R V E N T I O N

5

attract women and youth participation in the dairy sector. Youth are also likely to be

attracted by the digital technology promoted by the project. Additional business

opportunities for women and youth are expected to result from increased demand for

dairy inputs, which would support their entry into input supply businesses. Local value

addition to raw milk will be encouraged and, coupled with the project facilitating

linkages with local school feeding programmes, this is expected to enhance business

opportunities.

The project is located within the organisational structure of DAZ, which is a national

membership-based organization representing the dairy industry country-wide including

dairy producers, processors and dairy related agribusiness. It was registered under the

Societies Act in 2010. Its main objective is to promote and protect the interests of its

members. The project implementation team consists of a Project Manager; Manager -

Finance and Administration; Media Liaison Officer; Gender, Savings and Credit

Officer; Business Development Officer; and five Field Facilitators. The team is

supported by the DAZ Executive Manager, the Officer - Policy and M&E and the Dairy

Development Officer who provide oversight, project monitoring and extension support.

2.3 OBJECTIVES AND METHODOLOGY

In accordance with what is stated in the Terms of Reference (ToR), the review provides

conclusions and recommendations intended to help the Embassy of Sweden in Lusaka

to assess project progress and providing recommendations as a basis for their decision

on whether to continue funding the project. The review provides input to DAZ on how

to adjust and improve project implementation by identifying areas of possible

improvement, and recommendations for future programming. An evaluation matrix

with all evaluation questions, indicators and data sources is provided in Annex 2.

A multi-pronged approach has been used to collect and triangulate data from primary

and secondary sources. Written sources were reviewed to obtain an overview of the

project and to track progress. Documents reviewed include DIMS project documents,

including the project proposal, results framework, inception report and semi-annual and

other progress reports, budgets and financial reports. Sida documentation reviewed has

included the project appraisal, the project agreement and the strategy for development

cooperation with Zambia. Additional external documentation of relevance for the

project and its context was also reviewed, including policy, legal and institutional

frameworks, such as the 7th National Development Plan, the Vision 2030 and other

studies relevant to the dairy sector in Zambia. A list of literature reviewed is provided

in Annex 3.

The field work was participatory, involving DAZ, the Embassy of Sweden,

stakeholders and beneficiaries. An inception workshop with DAZ staff at the onset of

Page 16: Project Review of the Dairy Association of Zambia (DAZ)’s

2 T H E E V A L U A T E D I N T E R V E N T I O N

6

the evaluation process and a Force Field Analysis (FFA)4 workshop towards the end of

field work provided an opportunity for the project staff to express their priorities and

opinions, engage in forward looking strategic discussion and provide input to the

analysis and reporting of the evaluation team.

Key informant interviews were held with the Swedish Embassy, individual DAZ staff,

individual MCC staff in the provinces and other stakeholders, including those

suggested by Sida and DAZ. A list of persons interviewed is provided in Annex 4.

Based on the approach outlined in the inception report and taking into consideration

updated milk production data, eight out of the 32 MCCs covered by the project - two

each from the Central, Copperbelt, Lusaka and Southern Provinces - were selected for

field visits and enumeration. The list of MCCs covered is shown in Table 1 (next page).

Focus group discussions were conducted at all 8 MCCs. The focus group structure was

descriptive and evaluative. Dairy farmers were asked to express their perception and

feelings towards the DIMS project and the services it provides. The discussion was

moderated by a member of the evaluation team, with the aid of an interpreter where

necessary. After the discussion, a brief two-page questionnaire was administered to all

farmers present and willing to participate, both those profiled and not profiled5 on the

DIMS platform. The questionnaire is provided in Annex 5. A practical demonstration

of how the DIMS platform works was carried out by MCC staff at 3 of the 8 MCCs.

Although it was initially planned to target an average of 30 respondents at each of the

8 MCCs and to randomly select farmers for enumeration within the MCC, a purposive

sample was used for several reasons. January is a peak season for rainfed agricultural

activities in Zambia and the turnout of farmers for group discussion and interview was

low, especially since the planned meetings were within a time constrained window to

achieve coverage of 2 MCCs per day. Even in the one case where only one meeting

was scheduled for the day (Kayuni MCC), gathering farmers took a lot of time because

most came from afar and the rains on the day precluded timely attendance at the

meeting. A meeting scheduled for the morning took place in the afternoon when the

rain had abated. Despite high turnout at Magoye MCC, not all famers present at the

meeting filled out the questionnaires because they were either unwilling or unable to

do so. Given the time constraints and high illiteracy rates among members, it was not

possible for the consultant to assist all farmers needing help to complete the

questionnaire. From an attendance of 46 farmers, only 8 questionnaires had useful

4 Force Field Analysis is a group-based interactive tool with a dynamic orientation, carried out as a workshop where

key stakeholders specify factors that help and factors that hinder the achievement of the goals or objectives of an

organisation or intervention. 5 Profiling of milk suppliers means that they get entered into the DIMS system.

Page 17: Project Review of the Dairy Association of Zambia (DAZ)’s

2 T H E E V A L U A T E D I N T E R V E N T I O N

7

information and most were incomplete. Thus, apart from the sample being purposive,

this skewed further the representation from the individual MCCs.

A breakdown of the final sample of respondents is shown in Table 1. Out of the

respondents, 62% were male and the rest female. Only 7 out of the 100 were youth6.

Table 1 Survey sample characteristics

Name of MCC Total Sampled Male Female Youth

Central Province 17 16 1 0

Mpima 10 9 1 0

Mutaba 7 7 0 0

Copperbelt Province 29 8 21 2

Fisenge 16 2 14 2

Kwanshama 13 6 7 0

Lusaka Province 23 13 10 2

Mapepe 11 5 6 0

Palabana 12 8 4 2

Southern Province 31 25 6 3

Kayuni 23 17 6 3

Magoye 8 8 0 0

Grand Total 100 62 38 7

Sixteen DAZ staff members participated in the FFA workshop in which they identified

factors that help and hinder the achievement of the goal of the project. Each participant

wrote individually five helping and five hindering factors on post-it notes which were

collected, displayed and aggregated at levels from the individual and household level

up to overall policy and strategic factors at the top. The aggregated factors were listed,

one list for helping factors and one for hindering factors. Each participant was given a

print-out of both lists on which she/he noted down against each factor a score denoting

the importance of each factor according to his/her experience and assessment: ‘1’ for

the most important, ‘2’ for the second-most important, etc. The lists were collected and

again re-distributed to the participants so that each participant had one list with helping

factors and one with hindering factors. The scores for each factor were stated in plenary

by each participant and added to provide the total score for each factor, the factor with

the lowest score thus being the one considered the most important by the group.

6 Youth defined as 35 years old or younger.

Page 18: Project Review of the Dairy Association of Zambia (DAZ)’s

8

3 Findings

3.1 RELEVANCE

3.1.1 Is the project a technically adequate solution to the development problem at

hand in the dairy value chain? Does it eliminate the main causes of the problem the

sub-sector is facing?

The DIMS project is well aligned with the aspirations of Zambia’s development plans.

Zambia’s development agenda is guided by successive national development plans,

currently the 7th National Development Plan, 2017–2021, 7NDP. The Plan aims at

creating a diversified and resilient economy for the overall wellbeing of Zambian

citizens. The plan has five pillars, namely: i) economic diversification and job creation,

ii) reduced poverty and vulnerability, iii) reduced developmental inequalities, iv)

enhanced human development and v) conducive governance environment for economic

diversification.

The project was designed to address some of the key binding constraints identified in

the 7NDP, as they affect the dairy sector, including:

a) High cost and limited availability of long-term finance which affect the growth

of small and medium enterprises, including dairy smallholdings. These are to

be overcome by facilitating linkages to appropriate sources of finance and

formation of savings and credit groups;

b) Low labour productivity due to lack of physical capital, lack of access to

technology and weak human capital. This is to be addressed by reducing post-

harvest losses in milk production and increasing productivity through linkages

to capital providers;

c) Inadequate skills and innovation are to be addressed by capacity building of

smallholder dairy farmers and establishing an innovative digital system that

builds trust between smallholder dairy producers and service providers enabling

dairy producers to access necessary services to improve their businesses.

d) Weak/underdeveloped market information systems, which affects information

acquisition and the ease of doing business. This is to be overcome by

establishing a digital information system that disseminates information and

extension messages to augment existing extension services as well as reduce

the cost of farmers to access information.

When fully implemented, the DIMS is expected to alleviate some of the identified

problems within the project areas.

Page 19: Project Review of the Dairy Association of Zambia (DAZ)’s

3 F I N D I N G S

9

3.1.2 What is the relevance of the tools/instruments/approach used by the project

for creating demand for the DIMS services?

Apart from strong alignment with the 7NDP and the Strategy for Sweden’s

Development Cooperation with Zambia (as elaborated further in section 3.1.3), the

DIMS project is a timely innovation, given the digitization trend within the agricultural

sector in the country. Within the sector, it is in line with the country’s ongoing effort to

introduce e-governance. The project joins other programs like the Farmer Input Support

Programme (FISP) which administers smallholder farmer inputs through the e-voucher

system and links farmers to agro dealer services.

The DIMS approach has certain basic underlying assumptions for its interventions to

materialise as planned. These include that stakeholders and service providers will buy

into the DIMS model. On the beneficiaries’ part, access to a mobile phone and basic

literacy is essential, otherwise there is a risk that beneficiaries might not be able to make

full use of the DIMS products. Along the line of rail where most of the MCCs are

located, mobile network coverage is fairly good. In the baseline report for the DIMS

project, over 70% of the farmers reported fair to very good network coverage. A study

by the Zambia Information and Communication Technology Agency (ZICTA) in 2018

estimated that 54% of all individuals in the country were active users of mobile phones.

Usage is probably higher along the line of rail where MCCs are located. The major

limiting factor for DIMS usage is illiteracy among the older dairy farmers. This will

limit their access to the more advanced services planned by DIMS, for instance online

extension modules. Farmers can, however, still use the linkages to other services.

3.1.3 To what extent has the DIMS project conformed to the wider needs and

priorities of the project beneficiaries and the policies of the donor (Sida)?

Evidence from the field indicates that dairy smallholders access various services from

the MCCs, which the project will augment. Services accessed to improve productivity

and the rating by dairy farmers of the usefulness of these services is shown in Table 2.

The indicated figures represent farmers’ responses out of the total sample of 100.

Table 2 Access to, and rating of services at MCCs

Type of Service Number and % of

farmers accessing

(N=100)

Number and % of farmers rating

service as useful or very useful (

N=100)

1 Training 90 72

2 Extension 71 52

3 Credit 62 54

4 Bulk feed 59 49

5 Bulk medicines 41 31

6 Insurance 28 17

7 Equipment 27 22

Source: Questionnaire survey

Page 20: Project Review of the Dairy Association of Zambia (DAZ)’s

3 F I N D I N G S

10

Farmers were asked to prioritise the problems they encounter in dairy production. Items

indicated by at least 50% of the survey respondents included low milk prices, animal

disease, expensive feed, expensive veterinary drugs, lack of training and lack of

extension services. Among these problems, 60% of the farmers prioritised low milk

price as the most important or second most important problem; and 42% prioritised

expensive feed as the most or second most limiting factor. For all identified problems,

respondents were asked whether they felt that the DIMS project was addressing their

problems. For the two prioritised problems of low milk price and expensive feed, only

21% and 40% of the farmers felt that the DIMS project was intervening. The other

problems of expensive veterinary drugs, animal disease, lack of transport, lack of

extension services and lack of training had at most 20% respondents ranking them as

the most or second most limiting factors to increased production.

During focus group discussions, additional problems that featured prominently were:

lack of improved breeds of dairy cattle, lack of credit and inadequate access to clean

water resources, especially during the dry season.

It would appear that the project is targeting the right problems in as far as planned

linkages to suppliers including feed suppliers, and providing alternative dairy feed

resources via pasture cultivation. There is little evidence of intervention on the milk

price end, due to the monopolistic structure on the milk buyers and milk processors

side. In this case, given farmers’ minimal influence on milk price, the cost reduction

interventions that the project is undertaking, such as linking farmers to input suppliers

and service providers, and facilitating bulk buying of inputs and services, thereby

securing volume discounts and reducing costs, are well targeted.

Within the Strategy for Sweden’s Development Cooperation with Zambia (2018-2022),

the project falls under Strategic Area 3 which focuses on environment, climate,

renewable energy and sustainable inclusive economic development and livelihoods.

Relevant objectives in this area that speak to the 7NDP and implementation goals of

the project include sustainable productivity and production in agriculture, and increased

resilience to climate change; improved opportunities for sustainable livelihoods,

particularly for women and young people; and increased capacity to engage in

sustainable trade, access to markets and value chains for small-scale farmers,

entrepreneurs and businesses.

3.1.4 Is the project consistent with partners’ priorities and effective demand?

Does the project require external support from more experienced development

organisations? If yes, in what areas does DAZ need support? If not, how can DAZ

be more relevant to the local social and cultural conditions?

One clear result from the forcefield analysis workshop carried out with DAZ staff was

that dealing with climate change impact should be a priority for the project (see Section

3.2.2). While there has been some training in this regard, both to DAZ staff and milk

farmers, there is no staff member in the project with specific competence in this area.

Page 21: Project Review of the Dairy Association of Zambia (DAZ)’s

3 F I N D I N G S

11

Therefore, cooperation with external partners would be on option. During a focus group

meeting with staff of the Dutch development NGO SNV, who are in the process of

starting the implementation of a new Sida-financed project, INCREASE7, on

strengthening climate change resilience, it became clear that there is a strong interest

from their side to cooperate with the DAZ-DIMS. The opportunities for using DIMS in

their extension and dissemination work would be one important benefit. From the

perspective of DIMS, cooperation would be equally beneficial, notably in the areas of

climate-smart dairy business planning, fodder storage techniques and biodigester

technology for alternative energy production. The two projects already have budgets

for this type of work and could develop a joint work plan for their cooperation. Another

possible cooperation partner would be the Zambia Dairy Transformation Programme

(ZDTP) funded by Ministry of Foreign Affairs and Trade New Zealand.

Evidence from focus group discussions at MCCs suggests that while beneficiaries are

appreciative of the opportunity to improve their production and income by preventing

losses due to pilferage, milk adulteration, etc., between the farm and MCC, some

members are aware of the greater unrealized potential of the project and this has created

a crisis of expectations because of the slow implementation of these other aspects. For

example, farmers have complained that most of the promised linkages to suppliers and

service providers have not materialized.

It is found that the characteristics of MCCs are not homogeneous and the relevance of

and expectations from the current DIMS platform differ quite a lot among the MCCs.

For instance, at Palabana MCC, which is close to Lusaka, where dairy farmers are

generally well educated, either recently retired or currently still working in the city,

there is urgent demand for the digital part of the system to deliver more in addition to

real time notification of milk deliveries by SMS. Members have stated that a web-based

facility whereby they could access information from any computer anywhere rather

than just from the MCC would be required. On the other hand, as another example, at

Magoye MCC where there are a lot of members with generally low literacy levels and

where farmers are disillusioned because of a failed working relationship between MPS

- the original service provider - and the project, thereby affecting their enrolment to the

DIMS system, there was despondency as to what the DIMS project can actually

achieve. Here, there was more urgent need for the tangibles, including linkages to credit

sources, affordable feed, visible extension presence, etc. There is need, therefore, for

the project to do a quick needs assessment at each MCC, in order to tailor the services

to be provided and how these services should be phased in.

7 INCREASE: Increasing Climate Resilience in Energy & Agriculture Systems and Entrepreneurship

Page 22: Project Review of the Dairy Association of Zambia (DAZ)’s

3 F I N D I N G S

12

3.2 EFFECTIVENESS

3.2.1 How did the various capacity building approaches perform about achieving

the DIMS project objectives (including M&E initiatives)? What has been achieved?

The performance against indicators in the project result framework is provided in

Annex 6.

This table is included in the latest DAZ DIMS progress report, covering the period up

to 31 December 20198.

The table shows good progress on several of the important indicators, including:

- Profiling: from 0 beneficiaries at the outset to above 2,000 (end-of-project

target 2,600)

- Milk delivery information and e-extension is now provided through cellphone

messaging

- Average increase of lactation period increased from 200 to 280 days

- All MCCs capturing data are on the electronic platform

- 77 savings and credit groups formed and 334,552 ZK saved.

- Some, although limited, input and financial services access is provided

- Memoranda of Understanding with 13 suppliers, for instance farm inputs

such as feed, veterinary medicine or artificial insemination services.

With regard to the Memoranda of Understanding, while these have been entered into

there has been no major action yet under the agreements. Buy-in and onboarding of

processors and input suppliers has been slow. It is not clear from the project reporting

why this has been delayed but one probable contributing reason is that the project

spreads its resources too widely, instead of focussing on fewer MCCs and fewer

themes. Pricing of the service has been an issue and DAZ is facilitating fresh

negotiations for Vodacom to clarify the pricing mechanism and start with an

introductory price favourable to both parties which can later be renegotiated after

business expands.

It was found during the field visits to the eight MCCs in four provinces that some

beneficiaries are beginning to get disappointed with the rather limited services provided

so far. This is a serious problem since the interest and willingness among the

beneficiaries is important from demand and sustainability points of view.

The project results framework is of relatively good quality overall, but the indicator

definitions, mode of calculation and data sources need to be improved in order for M&E

8 DIMS: Semi - Annual Progress Report June - December 2019

Page 23: Project Review of the Dairy Association of Zambia (DAZ)’s

3 F I N D I N G S

13

to be more transparent on how numbers are arrived at and progress traced. This is

important so that anyone can be able to verify reported progress. Methods of analysis

and sources of data used need to be stated, and data collected from primary surveys

need to be stored and accessible for easy access by third parties who may wish to review

and verify. This is useful in case current M&E staff vacates and new personnel takes

over the function, as it would ensure a smooth transition and continuity of approach.

Comments on some selected indicators are provided in Annex 7 along with a generic

format for suitable indicator definition.

3.2.2 What are the reasons for good- or non-performance of the project?

The training programmes are considered successful and useful by the project and by

the beneficiaries, although there are complaints about the fact that they are not

followed-up by action or some kind of investment or demonstration. The reason the

project has been relatively successful here is that this is an activity they can control

themselves through their Field Facilitators. For most other outputs they depend on

service providers.

The results of the forcefield analysis workshop where 16 DAZ staff members identified

and ranked factors that help or hinder change, provides some additional factors for

success or non-success, see Tables 4 and 5.

Table 3 Force Field Analysis: Factors that support change

Rank Factors for change Score

1 Availability of a favourable policy environment, legislation

framework and governance system 58

2 Participants’ willingness to be part of the project 71

3 Availability of good infrastructure and well-functioning marketing

systems 80

4 Availability of supportive dairy industry developmental cooperating partners

82

5 Access to finance, input and equipment through private sector linkages

90

6 Availability of land and water 94

7 Capacity building of smallholder dairy farmers in livestock management, business and technical skills derived from research

97

8 Improved infrastructure for agricultural trade and growth 99

9 Increased women and youth participation in dairy 123

10 Improved governance at MCC level 128

11 Improved record keeping at MCCs and farmer level, which

enhances the M&E system 133

Page 24: Project Review of the Dairy Association of Zambia (DAZ)’s

3 F I N D I N G S

14

Table 4 Force Field Analysis: Factors that work against change

Rank Factors against change Score

1 Unfavourable climate change that negatively impacts on dairy

production 20

2 Smallholder dairy farmers incur high cost of production due to

high cost of finance, inputs and equipment 66

3 Inaccessibility to land and water 76

4 Limited milk market due to poor infrastructure off the line of rail - the infrastructural corridor linked by a rail line from the Copperbelt through Lusaka to Livingstone (roads, buildings, equipment and electricity)

85

5 Limited technical skills due to inadequate extension service provision to smallholder dairy farmers i.e. veterinary officers, reading materials and e-extension

92

6 Inadequate awareness creation and sensitisation of the national gender policy and unavailability of other legislation policy framework supporting the dairy value chain

101

7 High disease prevalence 103

8 Poor governance practices at cooperative level 127

9 Low literacy levels 128

10 Low adoption rate of new project concepts 138

11 Negative perception among the youth about agriculture in general

and dairy farming in particular 151

12 Negative cultural perception among the women that dairy farming

is a business for men 159

3.2.3 What can be done to make the project more effective?

The pilot project covers a large part of Zambia and almost half of all MCCs. Based on

the current status of DIMS development and the products that it currently generates, it

is assessed that the project spreads its resource too widely. Other indicators of this

include the lack of immediate training follow-up9 in terms of actual field action and

investment, the relatively limited presence of field facilitators at MCC level and the

limited progress on creating linkages to suppliers of dairy farming inputs and services.

The main purpose of the forcefield analysis workshop was to provide strategic input to

the planning of the project. It was well received by DAZ staff and, if followed by a

9 An example of this link between training and follow-up would be sensitization on the benefits of artificial insemination (AI)

and a demonstration on how it is done during one week. During the same week, immediately after the training or shortly after, it can be arranged to have AI technicians making appointments with those farmers who are ready to use the AI service and the

service delivered accordingly.

Page 25: Project Review of the Dairy Association of Zambia (DAZ)’s

3 F I N D I N G S

15

strategic planning exercise as intended, this could result in improved effectiveness in

the implementation of the project.

3.3 EFFICIENCY

3.3.1 Is the project staffing structure adequate to achieve the project objective?

The staffing structure of the project and the related staff expenditures are shown in

Annex 8. It is an appropriate structure as such, covering the main subject areas that the

project is working with. The staff have competence in the respective areas assigned to

them.

The staffing structure has the following three major limitations:

1. While the structure is top-heavy with many staff at the secretariat, the presence of

DAZ staff at the MCC level, notably the field facilitators, has been reported to be

relatively limited. The field facilitation will become even more important in the

future as the DIMS services develops.

2. There is no IT expert employed at the secretariat, who could facilitate the further

development and maintenance of the DIMS system and act as a technically

competent client counterpart to the service provider.

3. The organisation lacks specific competence in climate change mitigation,

adaptation and resilience and in the application of human rights based approaches

(HRBA), including poverty orientation.

3.3.2 Is the timing of the project favourable?

As mentioned earlier, this type of innovative IT solution to provide benefits for farmers

is timely in that there is interest among the beneficiary groups as well as the MCCs for

technical solutions that can improve management efficiency.

As mentioned under the relevance section, the project fits well with current national

and donor strategies.

The project is timely given Zambia’s focus to diversify away from mining to agriculture

and, within agriculture, to diversify the farmers’ portfolio according to their climatic

and comparative advantage. Also, demand for dairy products is increasing while most

processors are operating below capacity because of shortages of raw milk. Farmers can

cash in by increasing production and productivity.

The project potential is, however, to some extent limited by access to infrastructure -

mobile networks and phones - and farmers’ illiteracy that affects ability to adopt. Some

farmers will require more training themselves or that of their proxies in order to access

even the basic SMS milk delivery service. Training needs should be assessed for each

MCC individually, since they are all different with regard to membership, organisation

and capacity.

Page 26: Project Review of the Dairy Association of Zambia (DAZ)’s

3 F I N D I N G S

16

3.4 SUSTAINABILITY

3.4.1 Is it likely that the benefits of the project are sustainable and scalable? What

are the key lessons learnt so far? What actions are to be taken as a result?

Is it likely that the benefits of the project are sustainable and scalable?

The benefits of the project are potentially sustainable. Sustainability of benefits

depends to a large extent on whether the beneficiaries appreciate the benefits enough

to pay for the services after the external support to the DIMS project ends. Focus group

discussions at all MCCs acknowledged the benefits from SMS notification of milk

deliveries; i.e. reduced pilferage and milk adulteration by employees and the possibility

to monitor milk delivery and dairy activities even when one is away from the farm, record

keeping, etc. However, respondents expressed concern at the slow pace of

implementation on other aspects of what DIMS is supposed to deliver, mainly linkages

to suppliers and service providers. Survey findings indicate that 99% of the respondents

had used a DIMS tool and/or message function. Apparently even non profiled farmers

do get notifications of upcoming meetings from MCC staff.

At all 8 MCCs except Magoye, there was expressed willingness to pay for the services

provided, as long as members were informed about how much they would have to pay

and whether they could afford it. Only at Kayuni MCC did members recall a monthly

payment of 5 ZK per member per month having been discussed with DAZ, although

they said the deductions from their payments from milk deliveries to the MCC had not

yet been implemented. The DIMS project needs to develop and publicize an exit

strategy and ensure that all participating MCCs and their members are made aware of

their responsibilities to sustain the benefits after project end. Further, all costs to be

borne need to be clarified. A phased approach for takeover of the DIMS associated

costs by the system users is advised.

The case of Mutaba MCC, where the DIMS project severed the contract with MPS –

the initial service provider engaged by DAZ, but the MCC remains engaged with the

service provider inadvertently shows that it is possible for MCCs to sustain service

relationships independently, even without the DIMS project providing support. Some

members of the MCC said they were happy with the services MPS was providing.

The modalities of how this is working are subject to further investigation by DAZ and

some useful lessons could be learned. At Magoye MCC, another of MPS’s previous

clients, things were different. Members dissatisfied with the service from MPS were

not willing to have any additional deductions from their milk proceeds for non-existent

services. Only 10 out of about 50 participants in the focus group discussion at Magoye

indicated willingness to pay for the services after project end.

With regard to environmental sustainability, this is difficult to assess in the current

situation of climate change. On the one hand, an intended increase in the number of

Page 27: Project Review of the Dairy Association of Zambia (DAZ)’s

3 F I N D I N G S

17

cows will lead to more intensive natural resource use, which might have negative

environment impact and reduced sustainability. On the other hand, the expected socio-

economic impact of the project on household livelihoods would reduce poverty, which

can be expected to have positive impact on the environment as well as on sustainability.

The DIMS project appraisal assessed that DIMS had adequately considered

environmental aspects by its intention to promote climate-smart dairy farming

practices, safe use of veterinary drugs and increased savings which would build

farmers’ capital and enable investment in renewable energy products – solar and

biogas. This would decrease use of charcoal and fuel wood. DIMS did not conduct a

strategic environmental assessment, nor has an environmental impact assessment been

carried out.

With regard to climate-smart agriculture, DIMS has implemented demonstrations on

pasture cultivation and subsequently linked farmers to the Ministry of Fisheries and

Livestock for them to access drought resistant pasture and fodder seeds. By December

2019, 203 farmers had established pasture fields and 423 farmers had been trained on

feed mixing using local materials including the use of wilting materials in drought

prone areas. Additionally, 6 DAZ field facilitators had been trained on hydroponic

pasture production as mitigation for climate change and they will in turn train famers.

Overall, 20 percent of targeted farmers had adopted climate-smart practices, against a

target of 30 percent.

Although DIMS is actively pursuing farmer linkages to suppliers of veterinary drugs,

there is no evidence in the project reports that farmers are being trained on the safe use

of drugs, possibly because these linkages have not yet fully materialised.

Similarly, although DIMS has overachieved in the formation of savings clubs (77

against planned 30) and funds saved (ZMW 334,552 against planned ZMW 90,000),

there is no clear link between these numbers and investment into energy saving utilities.

There is no tracking of how the saved funds are applied for investment.

Scalability seems possible, if one assumes that the basic digital programming

infrastructure is already in place and additional MCCs coming on board will result in

incurring only minimal additional costs. This needs to be verified with the service

provider. Add-ons to be considered are both those to provide currently serviced MCCs

with additional functionalities as per needs assessment and the cost of adding on new

MCCs onto the platform. The DIMS project is not in a position to independently assess

the costings of additional offerings from the service provider and the cost of system

maintenance in the long-term, which they should factor into how costs to sustain the

system are apportioned in their exit strategy. In this respect, the project lacks an IT

specialist on the project team who is technically knowledgeable about the offerings

from the service providers in order for DIMS to interact with the service provider on a

more informed footing.

On the part of the DIMS implementation team, scalability relates to two fronts: scaling

up implementation of the existing platform to fully service members with linkages to

Page 28: Project Review of the Dairy Association of Zambia (DAZ)’s

3 F I N D I N G S

18

suppliers and service providers, uploading of extension modules and other promised

activities; and scaling out to additional MCCs. The project has so far implemented only

a limited number of functions of the system, which has created an interest among users

but will not be sufficient for sustainability in a longer time perspective. Once the project

perfects its act with the current 32 MCCs, it may even attract other funding to upscale

to a larger number of MCCs in case of Sida funding discontinuing, especially with

demonstrated sustainability of benefits in the current 32 MCCs.

3.5 CROSSCUTTING ISSUES

3.5.1 Has the project been implemented in accordance with the rights perspective:

i.e. have target groups been participating in project planning, implementations and

follow up? Has anyone been discriminated by the project through its

implementation? Has the project been implemented in a transparent fashion? Are

there accountability mechanisms in the project?

Have target groups been participating in project planning, implementation and

follow up?

Farmers at all 8 selected MCCs acknowledged that there had been adequate

sensitization about the DIMS project before implementation. However, their

involvement during project implementation appears to have been weak. Some farmers

expressed that their most immediate needs were not taken into account let alone being

addressed. At Mpima MCC, dairy farmers expressively indicated that they would have

preferred a needs assessment in which case they would have prioritized the completion

of their dairy processing plant. Farmers at Magoye and Kayuni MCCs complained

about lack of regular visits by field extension personnel and generally weak monitoring.

Thus there has been insufficient level of participation in project planning,

implementation and possible follow up activities. These findings, though anecdotal,

may be good pointers for the DIMS to reassess the current situation and reorient, if

necessary, the way they do business.

Has anyone been discriminated by the project through its implementation?

Fisenge MCC is an all-female entity, by design. Male participation is limited to

widowers or relatives of female members within the MCC. Men cannot ascend to

leadership positions. Thus there is discrimination against men. The group was initiated

by Heifer International, another organisation that focuses on the livestock value chain

in Zambia, as a women’s project which they granted dairy cows. Members have

preferred to maintain the female-only orientation because they feel they have more say

and control over their operations than if they were in a mixed group. Women at

Kwanshama who are in a mixed gender MCC are working on establishing an agro-

dealer shop. They expressed similar sentiments about the need to own something which

they can fully control themselves.

Page 29: Project Review of the Dairy Association of Zambia (DAZ)’s

3 F I N D I N G S

19

Has the project been implemented in a transparent manner?

In those MCCs that were serviced by MPS, there was seemingly a breakdown in

communication between DAZ and the MCCs to inform them about what was happening

and what remedies DAZ was taking to remedy the situation. Continuous

communication is important in order to build and maintain trust between project staff

and beneficiaries. At MCC level, there was some evidence of governance issues. In

some cases MCCs were not able to discuss freely the status. Anecdotal evidence

suggests that there were problems at Kwanshama and Palabana MCCs. At Kwanshama,

the last annual general meeting was held in 2018. At Palabana, the last cleared audit was

done in 2006. The audit of 2016 was rejected by members.

Accountability mechanisms at DAZ are available through Board oversight, donor

monitoring and financial audits. These mechanisms are currently considered adequate.

However, DAZ general operations - especially for cross-cutting areas such as gender

and M&E - are sometimes intricately intertwined such that it becomes difficult to

account for time for each part. For example, DAZ is generally interested in increasing

milk yields per cow per day. The DIMS project might target specific activities to this

end, but monitoring the milk yield trend is of interest to both the DIMS project and

DAZ in general. Detailed time accounting on time sheets is one partial remedy but does

not solve the problem of the staff being unsure of whether the work is done for DAZ or

DIMS.

3.5.2 Has the project had any positive or negative effects on gender equality?

Could gender mainstreaming have been improved in planning, implementation or

follow up?

Has the project had any positive or negative effects on gender equality?

The level of participation by gender at general membership and leadership levels were

used to examine gender equality. The level of participation of men, women and youth

in dairy activities is predetermined by the culturally defined low levels of ownership of

cattle by women and the lack of capital by youth to purchase their own cattle. Further,

youth generally perceive dairy farming as an occupation for old people (see Section

3.5.3). Out of the total membership of 3038 in 32 MCCs, 859 (28%) are female (Source:

DAZ DIMS MCC listings).

Table 5 shows that out of a total membership of 1015 in the 8 selected MCCs, 289

(28%) were women and 381 members (38%) had enlisted on the DIMS platform. Of

the 381 profiled members, 31% were female. Apart from the generally slow pace of

enrolment on to the DIMS platform, there seems to be equitable recruitment onto the

system, with 31% of those enrolled being female against the female proportion of 28%

in the population of the 8 MCCs. The situation as of 2019 year-end reporting shows a

similar pattern, with 2279 members profiled across the 32 MCCs of whom 523 (23%)

were women (DIMS Semi Annual Report, June-December, 2019). Female participation

in MCCs ranged between 2% at Mungaila MCC and 63% at Kampelembe. Two

Page 30: Project Review of the Dairy Association of Zambia (DAZ)’s

3 F I N D I N G S

20

exceptions both in the Table 5 and in the semi-annual report are Fisenge MCC, the

women-only MCC, with 95% female participation at year end and Mutaba MCC with

no activity because of severance of agreement between DAZ and MPS, the service

provider.

Table 5 Participation in DIMS by gender as on 21-1-2020

Name of MCC Membership Profiled Membership

Total Male Female Female %

Total Male Female Female %

Central Province

Mpima 56 45 11 19.6 27 22 5 18.55

Mutaba 13 12 1 7.7 0 0 0 0.0

Copperbelt Province

Fisenge 145 5 140 96.6 29 3 26 89.7

Kwanshama 40 23 17 42.5 40 23 17 42.5

Lusaka Province

Mapepe 61 40 21 34.4 57 36 21 36.8

Palabana 104 76 28 26.9 76 55 21 27.6

Southern Province

Kayuni 60 50 10 16.7 32 27 5 15.6

Magoye 536 475 61 11.4 120 96 24 20.0

All Selected MCCs 1015 726 289 28.5 381 262 119 31.2

Source: DIMS MCC membership listings as at 21.1.2020

Women are fairly represented at leadership level in the MCCs with those on the Board

generally occupying less powerful positions at committee level. Out of 269 Board

members at 32 MCCs, 83 or 31% are female. This is not bad considering that women

constitute 29% of MCC membership. Twenty one MCCs or 66% had at least 29%

female representation on their Boards. There are, however, a few exceptions in terms

of positions occupied by women. Among the MCCs surveyed the female-only MCC of

course had a female chairperson. Kayuni MCC also had a female chairperson. At

Kwanshama MCC a young female was part of the Board.

Female participation in the DIMS project is otherwise expressed in Savings and Credit

Groups where most members are female. DAZ has facilitated formation of 77 groups

consisting of 846 members of whom 506 (60 %) are women, 13% are men and 27%

are youth. At Kwanshama MCC, one of those selected for field survey, female members

of the MCC are in the process of establishing an agro dealer shop to deal in animal

health products or veterinary medicines.

Could gender mainstreaming have been improved in planning, implementation or

follow up?

A gender policy is in place at DAZ. It was finalized in July 2019 and a validation

workshop was held in November 2019, but it is yet to be ratified by the Board. The

Page 31: Project Review of the Dairy Association of Zambia (DAZ)’s

3 F I N D I N G S

21

process to institutionalize a gender policy has been lengthy, from 2018. In the absence

of ratification, implementation of activities has been held up. Some of the activities

could have been incorporated in previous work plans and approved individually by the

Board.

The gender policy outlines several strategies and measures and activities which need to

be undertaken to achieve gender mainstreamed operations and inclusive development.

Similarly, the Gender Audit of 2019 made several recommendations to address some

of the hindrances to progress in the dairy sector. The measures and recommendations

need to be prioritized in the case of DIMS and focus should be on a few areas that can

be feasibly addressed as not all measures can be implemented during the life of the

project. A quick review of the documented measures and activities in the Gender Policy

and the work plans of the Gender Specialist during the last 6 months of 2019 reveals a

weak link between the policy on the one hand and the work plan on the other, with

respect to activities implemented thus far. Hence, there is need to institutionalize and

adopt the Gender Policy; and synchronize the policy with work plan activities for

implementation.

A measure adopted by the DIMS project on their work in association with We Effectis

the Fair Resource Allocation System (FRAS), an indicator for equitable allocation of

resources between men and women for which it is required that at least 50% of the

resources should be allocated to women. Among the weaknesses highlighted by the

Gender Audit (2019) is that there are no internal mechanisms to ensure FRAS is fully

implemented. Notably, the FRAS indicator is not included in the most recent semi-

annual report of the DIMS project. It needs to be visible among the mainstream results

framework, if indeed it is one that should be tracked. According to the January-

December 2019 DAZ report for WeEffect, the project achieved 20% allocation to

women’s activities.

3.5.3 Has the project contributed to engaging more youth in the dairy value

chain? How can the project increase their engagement of youth in the dairy value

chain effectively and sustainably?

Has the project contributed to engaging more youth in the dairy value chain?

There is minimal participation by youth in DIMS mainstream dairy activities. Dairy

farming is perceived to be an old peoples’ occupation, a finding that has been supported

by evidence from focus group discussions with MCCs in the four provinces visited, as

well as interviews with DIMS staff and the results of the forcefield analysis workshop

carried out at DAZ. In addition, young people lack the capital with which to purchase

dairy cattle to start a dairy enterprise.

Youth10 do not constitute a sizeable part of the membership of MCCs. They were

virtually absent during focus group discussions in the 8 MCCs. Notably, two youths

that were involved in milk production participated in discussions at Fisenge and

Kwanshama MCCs. In the case of the young lady at Kwanshama, she inherited her

10 Persons aged 35 years and below

Page 32: Project Review of the Dairy Association of Zambia (DAZ)’s

3 F I N D I N G S

22

dairy cows from an aging mother. The 11-year old male youth at Fisenge acquired the

start-up dairy cow through the Heifer pass-on program from a sister, herself having

been gifted with a pass on from the grandmother.

There is higher participation of youth in savings and credit groups. Out of the 846

members in 77 groups, youths formed 27%. Youths have their own exclusive groups

where they launch their activities, including chaff cutting using machinery supplied by

the DIMS project. Chuff cutting business by youth was ongoing at Kwanshama and

Fisenge. The extent to which youths can engage in other value addition activities along

the dairy value chain is limited by the volume of savings that they can draw upon to

invest in these activities.

How can the project increase their engagement of youth in the dairy value chain

effectively and sustainably?

Youths lack medium to long-term capital to invest in businesses. Savings and Credit

Groups, even though they have less stringent terms of lending, usually lend money on

short-term basis. The DIMS project can assist the better organized youth groups to

develop viable business plans and link them with other youth-oriented funding to help

them grow their businesses. Youths also need to be sensitized on the other opportunities

in the dairy value chain apart from milk production. Available opportunities will differ

in each locality.

3.5.4 Has the project had any positive or negative effects on the environment?

Has the project had any positive or negative effect on the beneficiaries' resilience in

terms of climate change such as drought? How could the project improve its

planning, implementation or follow up on activities?

There are no indications of any negative effects on the environment caused by the

implementation of the project.

With regard to climate-change resilience, although this poses a serious threat to the

sustainability of milk production it is an area where the project has done little until now.

Current values of indicators on the production of milk per cow show a declining trend,

the explanation given being the droughts that have occurred during recent years. The

potential cooperation with organisations such as SNV could assist in alleviating such

effects.

The project aims at, and has already achieved to some extent, improvements of the

livelihoods of milk farmer households, which would have enhanced their resilience to

climate change and extreme weather events, including drought, since they now have

larger margins in terms of household income and resources.

As mentioned earlier, the project cooperating with other organisations with technical

capacity in climate-change resilience, could assist in tackling environmental and

resilience-related issues in an effective way. This does not mean, however, that the

project should not acquire that type of competence in its organisation, but this needs to

be done in a way that does not further increase the high cost of project staff at DAZ

Page 33: Project Review of the Dairy Association of Zambia (DAZ)’s

3 F I N D I N G S

23

headquarters. Neither should it mean that the environment and climate-change related

work is handed over to external implementor. The cooperation should build on a joint

work plan (see 3.1.4), the results of which would benefit both organisations in terms of

concrete physical outputs as well as capacity building in both organisations and among

the project beneficiaries.

Page 34: Project Review of the Dairy Association of Zambia (DAZ)’s

4 Evaluative Conclusions

The following are the main evaluative conclusions:

- The project is highly relevant. It fits well with relevant strategic and policy

documents and should, if successful, provide substantial benefits to its stakeholders

and intended beneficiaries.

- Judging from the newly reported current indicator values, the project has improved

its effectiveness during the later stage of implementation. However, the current size

of the pilot is too large to allow for optimal effectiveness, the project would benefit

from being scaled down and producing more tangible results to DIMS users rather

than scaled up.

- The project and its results can potentially be sustainable since there is willingness

among farmers to pay for the system as long as the costs are clarified and its value

clearly demonstrated. However, this requires that the project is able to show more

benefits emanating to the beneficiaries from the actual DIMS than what is currently

the case.

- It is concluded that while there have been some outputs that should benefit

environmental sustainability, including demonstration and training related to

pasture management and fodder, no strategic environmental assessment has been

carried out.

- While the structure is top-heavy with many staff at the secretariat, the presence of

DAZ staff at the MCC level, notably the field facilitators, has been reported to be

relatively limited. The field facilitation will become even more important in the

future as the DIMS services develops and this may mean that the distribution of

staff capacity between secretariat and field staff needs to be reassessed. This is the

case also with regard to the current professional profiles and competence in

emerging themes such as climate change and HRBA.

- Action to adapt to climate change is a priority theme for DAZ to address and this

should be done through cooperation with partners. The evaluation team concludes

that SNV would be a suitable partner in this regard. Another potential cooperation

partner would be ZDTP. To the extent that in-house competence in such areas is

deemed necessary and acquired, this needs to be matched with a corresponding

reduction in staff with different profiles.

- While the project will serve to increase benefits for people living in poverty

following from its provision of services to smallholder dairy farmers, there is no

strategy for how to reach the most marginalised farmers or how to make the services

useful for illiterate people.

- Although a gender policy has been prepared, this has not yet been ratified or made

operational.

- Youth and women cannot fully participate in mainstream dairy farming activities

Page 35: Project Review of the Dairy Association of Zambia (DAZ)’s

4 Evaluative Conclusions

because few own dairy cattle, most youth are not interested in dairy farming and

both women and youth are interested in other activities along the dairy value chain 23

but lack resources for investment.

Page 36: Project Review of the Dairy Association of Zambia (DAZ)’s

5 Recommendations

24

5.1 RECOMMENDATIONS TO DAZ

The following recommendations are made to DAZ:

Policy

- The DAZ Board should ratify the gender policy to enable DAZ to implement the

activities suggested in the policy. An action plan for implementation should

prioritize activities based on what can feasibly be attained within the timeframe

of the project and also considering previous recommendations from the gender

audit of 2019.

Strategy

- Follow up the results of the forcefield analysis workshop with a strategic planning

activity aiming at improving project implementation effectiveness

- Speed up implementation related to finalising profiling, developing the system to

meet the more advanced needs of the beneficiaries, and providing inputs under the

Memoranda of Understanding in order not to lose the interest of the beneficiaries

- In order to achieve the previous point, scale down and intensify rather than scale up

and out, to the extent this is possible keeping in mind that the system service

provider needs to have a certain volume of business.

- The DIMS project needs to produce and publicize an exit strategy and ensure that

all participating MCCs and their members are made aware of their responsibilities

to sustain the benefits after project end. Further, all costs to be borne need to be

clarified. A phased approach for takeover of the DIMS platform and associated

costs is advised.

- Carry out a strategic environmental assessment and prepare an action plan for how

the project should work with environment and climate change mitigation, adaptation

and resilience, including how to cooperate with partners. Indicators for

implementation of such a plan should be included in the project result framework,

and the implementation of the plan should have its own budget line in the project

budget.

- The project should prepare a strategic action plan for how to reach the more

marginalised farmers. This could be done by applying an Ex Ante Poverty Impact

Assessment (PIA) methodology11, through which the project can target people

11 http://www.value-chains.org/dyn/bds/bds2search.details2?p_phase_id=645&p_lang=en&p_phase_type_id=2

Page 37: Project Review of the Dairy Association of Zambia (DAZ)’s

5 R E C O M M E N D A T I O N S

25

living in acute poverty specifically and more directly, as a complement to the regular

activities addressing rural poverty in a more general sense. Indicators for

implementation of such a plan would need to be included in the project result

framework, and the implementation of the plan shall be covered by a specific budget

line in the project budget.

- To increase the participation of youth and women, a separate sub-component fo-

cusing on youth and women empowerment to develop small businesses should be

considered. Activities under this component should include capacity building for

these groups to identify and develop business plans for projects of interest along the

value chain, training on business management and leadership, and provision of start-

up capital or linkage to special funds for women and youth to obtain start-up capital.

Technical and planning

- The measures and recommendations need to be prioritized and focus should be on a

few areas that can be feasibly addressed as not all measures can be implemented

during the life of the project.

- Conduct a needs assessment at each MCC and prioritize what is to be delivered in

the remaining project period and a continued phase

- Engage constantly with the beneficiaries. An example of such interaction could be

facilitated quarterly self-evaluation/lesson sharing events at MCC level.

- The project needs to engage an IT specialist on the project team who is technically

knowledgeable about the offerings from the service providers in order for DIMS to

interact with the service provider on a more informed footing.

- Intensify training, based on training needs assessment for each MCC specifically,

combined with relevant follow-up activities in the field.

- To address the limiting factor of illiteracy, the inclusion of audio modules of

extension that illiterate farmers can access at cooperative level should be

considered. With the help of field facilitators some lead farmers should be

identified who can help organize an orderly way to access this audio information.

- Enhance the presence of extension workers at the MCCs by way of additional

recruitment in some MCCs

- Initiate discussion with SNV on content of cooperation related to their project

INCREASE. Discussions could also be initiated with the ZDTP to develop potential

cooperation.

Page 38: Project Review of the Dairy Association of Zambia (DAZ)’s

5 R E C O M M E N D A T I O N S

26

5.2 RECOMMENDATIONS TO EMBASSY OF SWEDEN

The following recommendations are made to the Embassy of Sweden:

- Answering directly to the purpose of the review, it is recommended that Sida

continue supporting a continued implementation of the project, in line with the

above recommendations to DAZ.

- The remaining part of the current implementation period should be used, in addition

to implementation as recommended above, for preparation of a new three-year

project phase.

- Sida should facilitate that DAZ-DIMS can enter into a close cooperation with the

new SNV project on environment and climate-change mitigation, adaptation and

resilience, financed by Sida. Budgets that could cover such activities are already

available in both projects and no additional project or sub-project would be

required.

Page 39: Project Review of the Dairy Association of Zambia (DAZ)’s

27

Annex 1 – Terms of Reference

Page 40: Project Review of the Dairy Association of Zambia (DAZ)’s

A N N E X 1 – T E R M S O F R E F E R E N C E

28

Page 41: Project Review of the Dairy Association of Zambia (DAZ)’s

A N N E X 1 – T E R M S O F R E F E R E N C E

29

Page 42: Project Review of the Dairy Association of Zambia (DAZ)’s

A N N E X 1 – T E R M S O F R E F E R E N C E

30

Page 43: Project Review of the Dairy Association of Zambia (DAZ)’s

A N N E X 1 – T E R M S O F R E F E R E N C E

31

Page 44: Project Review of the Dairy Association of Zambia (DAZ)’s

A N N E X 1 – T E R M S O F R E F E R E N C E

32

Page 45: Project Review of the Dairy Association of Zambia (DAZ)’s

A N N E X 1 – T E R M S O F R E F E R E N C E

33

Page 46: Project Review of the Dairy Association of Zambia (DAZ)’s

A N N E X 1 – T E R M S O F R E F E R E N C E

34

Page 47: Project Review of the Dairy Association of Zambia (DAZ)’s

A N N E X 1 – T E R M S O F R E F E R E N C E

35

Page 48: Project Review of the Dairy Association of Zambia (DAZ)’s

36

Questions raised in ToR Indicators to be used in the review Methods Sources Availability and reliability of

data /comments

Relevance

Is the project a technically adequate solution to the development problem at hand in the dairy value chain? Does it eliminate the main causes of the problem the sub-sector is facing?

Ranking of the solution among other alternative technical solutions for feasibility and practicality in solving identified problems, which include14:

- low productivity

- scarcity of extension services - poor animal husbandry practices - limited access to, finance, improved dairy breeds,

water and key supplementary feeds - poor access to market for raw milk, especially

during the peak production periods - low levels of entrepreneurship and commercial

orientation amongst smallholder dairy farmers

Extent to which underlying assumptions for the solution to work are attainable and realistic

Document analysis

Focus group discussion

Force-field analysis

Interviews

Project results framework

Review of dairy sub-sector studies

Discussion with project staff and key stakeholders

Interview with independent player in the dairy value chain - key informant/ academic

The underlying assumptions to achieve results might themselves will be reviewed as part of the Force-Field Analysis and updated if the project

implementers find the need to do so.

What is the relevance of the tools/instruments/approach used by the project for creating demand for the DIMS services?

Dairy small holder capacity to use various tools/ instruments / approaches used by the project

Ease of access to and use by targeted clients

Document analysis

Focus group discussion

Interviews Field observation

Questionnaire survey

Related academic studies on the tools/ instrument s/approaches used

Review of similar project documents and their experiences with the tools/instruments/approaches

Studies and project documents of similar projects will be reviewed, where available

Quantitative data

14DAZ Project Proposal to Sida, April 2018, pp. 4-5.

Annex 2 – Evaluation Matrix

Page 49: Project Review of the Dairy Association of Zambia (DAZ)’s

A N N E X 2 – E V A L U A T I O N M A T R I X

37

MCC client perception of relevance of the tools/

instruments/ approaches

Actual level of module use among beneficiaries

Group discussion with Implementers and MCC clients

Questionnaire survey issued to participants in MCC focus group meetings

(in %) can probably be drawn from the questionnaire survey

Actual usage of module, e.g. information on milk delivery, events or training

Get extension assessment report

Baseline study

Where applicable and possible, MCC members and MCC staff at selected MCCs should demonstrate how to use some of the tools/ instruments.

To which extent has the DIMS project conformed to the wider needs and priorities of the project beneficiaries and the policies of the donor (Sida)?

Alignment with national policies and strategies

Alignment with Swedish development cooperation policy and strategy

Perception of relevance among project beneficiaries

Document study

Interviews

FGDs

Questionnaire survey

Relevant national policy and strategy documents, including the National Development Plan for 2017-2021 and Vision 2030

DAZ strategic plan

All required documentations is assessed to be available for this analysis

SDGs (7 of them)

Sweden’s strategy for development cooperation with Zambia

Is the project consistent with partners’ priorities and effective demand? Does the project require external support from more experienced development organisations? If yes, in what areas does DAZ need support. If not, how can DAZ

Stakeholder and beneficiary perceptions of consistency with their stated priorities15

Document study

Interviews

FGDs

Questionnaire survey

Focus group meetings with DAZ

Interview with Embassy staff

FGD with project stakeholders and beneficiaries

-

15See updated list of stakeholders and beneficiaries on pp. 4-5

Page 50: Project Review of the Dairy Association of Zambia (DAZ)’s

A N N E X 2 – E V A L U A T I O N M A T R I X

38

be more relevant to the local social and cultural conditions?

Effectiveness

How did the various capacity building approaches perform about achieving the DIMS project objectives (including M&E initiatives)? What has been achieved?

Reported project achievement levels

Selected achievement levels verified by stakeholders and beneficiaries

Document study

Interviews

FGDs

Progress and financial reports

Interviews, workshops and meetings with DAZ and Embassy

Stakeholders and beneficiaries

-

What are the reasons for good- or non-performance of the project?

Strengths and weaknesses observed by stakeholders and beneficiaries

Document study

FGDs

Interviews

Questionnaire survey

Progress and financial reports

Interviews, workshops and meetings with DAZ and Embassy

Questionnaire survey with project beneficiaries

-

What can be done to make the project more effective?

Weaknesses and opportunities related to project planning and implementation

Document study

Interviews

FGDs

Force-Field Analysis

Questionnaire survey

Progress and financial reports

Interviews, workshops and meetings with DAZ and Embassy

Efficiency

Is the project staffing

structure adequate to achieve

the project objective?

Project organisational structure

Number, designations and professional profile and

competence of project staff

Availability of competence in key emerging issues, e.g. climate change mitigation, adaptation and resilience; and HRBA

Document study

Interviews

Analysis of project management

information including number and

competences of staff.

-

Is the timing of the project favourable?

Project in relation to policy and macro-economic changes; technology development; and changes in product demand and other value-chain related changes.

Document study

Interviews

Questionnaire survey

Policy documents and macro-economic studies

Project reports

Value-chain studies carried out by stakeholders

Questionnaire survey with project beneficiaries

Sustainability

Is it likely that the benefits of the project are sustainable

Perception of the relevance of actual and intended project results among stakeholders and beneficiaries.

Document analysis

Focus group discussion

Project documents, minutes at meetings, records of planning

Results can be quantified,

Page 51: Project Review of the Dairy Association of Zambia (DAZ)’s

A N N E X 2 – E V A L U A T I O N M A T R I X

39

and scalable? What are the key lessons learnt so far? What actions are to be taken as a result?

Availability, realism and application of project exit strategy

Scalability to cover remaining MCCs

Interviews

Questionnaire survey

workshops; etc.

Key informants from among the MCC membership and DAZ staff

Review of MCC constitutions

Stakeholders

Group discussion with MCC leadership

Exit strategy/sustainability plan

expressed in percentages (as per answers by the respondents of the survey) and triangulated by the other information sources.

Crosscutting issues

Has the project been implemented in accordance with the rights perspective: i.e. Have target groups been participating in project planning, implementations and follow up? Has anyone been discriminated by the project through its implementation? Has the project been implemented in a transparent fashion? Are there accountability mechanisms in the project?

Members’ awareness about and involvement in the planning process at selected stages of project implementation

Extent of members’ participation in the planning process, as evidenced by attendance register at pertinent meetings

Inclusiveness of joining/membership criteria by gender, age, disability, etc.

Existence of independent accountability mechanisms and examples of penalty enforcements

Document analysis

Focus group discussion

Interviews

Observation

Questionnaire survey

Project documents, minutes at meetings, records of planning workshops; etc.

Key informants from among the MCC membership

Processors

Review of MCC constitutions

Group discussion with MCC leadership

Gender assessment for the diary sector

Additional secondary information sources to be identified by DAZ

Selected MCCs and project staff to prepare required documents

Has the project had any positive or negative effects on gender equality? Could gender mainstreaming have been improved in planning, implementation or follow up?

MCC membership by gender before and at the time of the review

Structure of MCC leadership (board, secretariat, cooperative) by gender before and after the project intervention

Gender policies and manuals before and after the project

Document analysis

Interviews

Observation

Baseline study, disaggregated project data on structure of membership at beginning of project and currently

Key informant interview with project staff, especially the Gender, Savings and Credit Officer

Interview with the leadership of the selected MCCs

Gender policies and manuals

Documents with We Effect

In case where baseline study did not collect detailed disaggregated data, data will be obtained from the MCC membership records.

It is assumed that, at the least, selected MCCs will extract and provide disaggregated data as required, if not already available

Page 52: Project Review of the Dairy Association of Zambia (DAZ)’s

A N N E X 2 – E V A L U A T I O N M A T R I X

40

Extent of adherence to FRAS16

Has the project contributed to engaging more youth in the dairy value chain? How can the project increase their engagement of youth in the dairy value chain effectively and sustainably?

MCC membership by age before and at the time of the review

Structure of MCC leadership by age before and after the project intervention

Extent of youth participation in other dairy value - adding activities, e.g. transportation of milk or input supplies, overseen by the project/MCC

Number of youth clubs before and after

Document analysis;

Interviews; Observation

Baseline study, project data on structure of membership and ongoing dairy value-adding initiatives at all participating MCCs

MCC records

Stakeholders

Membership data

In case where baseline study did not collect detailed disaggregated data, data will be obtained from the MCC membership records.

It is assumed that, at the least, selected MCCs will extract and provide disaggregated data as required, if not already available

Has the project had any positive or negative effects on the environment? Has the project had any positive or negative effect on the beneficiaries' resilience in terms of climate change such as drought? How could the project improve its planning,

implementation or follow up on activities?

Extent of MCC member awareness about climate change

MCC member perceptions about how their project activities might affect the environment, negatively and positively and mitigation measures they might take in case of negative consequences

Evidence/ examples of specific project interventions aimed at building resilience and avoiding negative environmental impacts

Document analysis;

Interviews; Observation,

Focus Group Discussions

Questionnaire survey

Project annual and semi-annual reports

Interview with project staff

Discussion with MCC members and stakeholder

_

16 FRAS: Fair Resource Allocation System, which requires that at least 50% of the resources should be allocated to women.

Page 53: Project Review of the Dairy Association of Zambia (DAZ)’s

Annex 3 – Documents studied

41

Published by Title Year

DAZ Progress Reports June-December 2019,

June 2018-May 2019

2018,2019

DAZ DIMS/DFS Project Proposal 20180403 2018

DAZ Gender Policy 2019

DAZ e-Consultancy Needs Assessment 2018

DAZ Strategic Plan 2017-2022 2018

DAZ DIMS Financial Report June 2018-

December 2019

2020

DAZ-DIMS Baseline report

FAO Food Outlook, November 2013 2013

FAO Food Outlook, 2019, Biannual Report on

Global Food Markets

2019

Government of

Zambia

7th National Development Plan, 2017–

2021

2017

Hofer, Anna Small Scale Dairy Farming in Zambia 2015

Ministry for Foreign

Affairs

Strategy for Sweden’s Development

Cooperation with Zambia 2018-2022

Embassy of Sweden,

Lusaka

Grant Agreement between Embassy of

Sweden and Dairy Association of Zambia

2018

Embassy of Sweden,

Lusaka

Appraisal of Intervention DAZ 2018

SNV INCREASE: Increasing Climate Resilience

in Energy & Agriculture Systems and

Entrepreneurship, Project Proposal

2020

WeEffect DAZ ODA Action Points

WeEffect DAZ Organizational Assessment

World Bank What would it take for Zambia’s

beef and dairy industries to

achieve their potential?

2011

Page 54: Project Review of the Dairy Association of Zambia (DAZ)’s

Annex 4 – Persons interviewed

42

Designation

DAZ staff, Lusaka

Executive Manager

Project Manager

Policy M&E Manager

Finance &Administration Manager

Dairy Development Officer

Gender, Savings And Credit Officer

Media Liaison Officer

Business Development Officer

Office Manager

Accounts Assistant

Field Facilitator

Field Facilitator

Field Facilitator

Field Facilitator

Intern

Intern

Lusaka

Swedish Embassy

Swedish Embassy

We Effect

Coordinator Renewable energy, SNV

Technical Officer renewable energy, SNV

Sector leader Agriculture/PM, SNV

PME Officer, SNV

Sweden

Swedish Milk Farmer and “donor”

to We Effect

Name Sex Designation

Persons Met At MCCs In Provinces

A Kayuni Mcc

1 F Member

2 F Member

3 F Member

4 F Treasurer

5 M Board Member

6 M Member

7 M Vice Chairperson

Page 55: Project Review of the Dairy Association of Zambia (DAZ)’s

A N N E X 4 – P E R S O N S I N T E R V I E W E D

43

8 M Board Secretary

9 M Member

10 M Member

11 M Member

12 M Member

13 M Member

14 M Member

15 M Member

16 M Member

17 F Board Member

18 F Member

19 M Member

20 M Member

21 F Board Chairperson

22 F Member

23 M Member

24 M Member

25 M Member 8=F,

17=M

B Kwanshama Mcc

1 F Member

2 M Secretary

3 M Chairperson

4 M Member

5 F Member

6 F Member

7 F Member

8 F Member

9 M Member

10 M Member

11 M Member

12 M Member

13 F Member

14 F Member

15 F Member

16 F Mcc Milk Tester

17 F Mcc Manager F =10;

M=7

C Fisenge Mcc

1 F Member

2 F Member

3 F Member

4 F Member

5 F Board Treasurer

6 F Member

7 M Member

8 F Member

9 F Member

10 F Member

11 F Member

12 F Board Vice Chairperson

13 F Board Chairperson

Page 56: Project Review of the Dairy Association of Zambia (DAZ)’s

A N N E X 4 – P E R S O N S I N T E R V I E W E D

44

14 F Member

15 M Member

16 F Member F=14;

M=2

D Mutaba Mcc

1 M Board Member

2 F Milk Attendant

3 F Mcc Manager

4 M (R) Member

5 M Vice Chairperson

6 M (R) Member

7 M (R) Member

8 M (R) Member

9 M Board Member

10 M Member

11 M Member F=2; M=9

E Mpima Mcc

1 M Member

2 M Member

3 M Member

4 M Mcc Accountant 5 M Member

6 F Member

7 M Member

8 M Member

9 M Vice Chairperson

10 M Vice Secretary

11 M Member

12 M Member F=1;

M=11

F Palabana Mcc

1 M Board Treasurer

2 M Board Member

3 M Vice Secretary

4 F Board Secretary

5 M Board Member

6 F Member

7 M (R) Member

8 M Board Chairperson

9 M Board Vice Chair

10 M Board Member

11 M Board Member

12 F Member F=3; M=9

G Magoye Mcc

1 F Member

2 M Member

3 M Member

4 F Member

5 F Member

6 M Member

7 M Member

Page 57: Project Review of the Dairy Association of Zambia (DAZ)’s

A N N E X 4 – P E R S O N S I N T E R V I E W E D

45

8 F Member

9 F Member

10 M Member

11 M Member

12 M Member

13 M Member

14 M Member

15 M Member

16 M Member

17 M Member

18 M Member

19 M Member

20 M Member

21 M Member

22 M Member

23 M Member

24 M Member

25 M Member

26 F Member

27 F Member

28 M Member

29 F Member

30 M Member

31 M Member

32 M Member

33 M Member

34 M Member

35 M Member

36 M Member

37 M Member

38 M Member

39 M Member

40 M Member

41 M Member

42 M Member

43 M Member

44 M Member

45 M Member

46 M Member F=8;

M=38

H Mapepe Mcc

1 M Member

2 M Member

3 M Member

4 M Board Secretary

5 F Board Member

6 F Member

7 F Member

8 F Member

9 F Member

10 M Member

11 M Board Chairperson

12 F Board Vice Chairperson F=5; M=7

Page 58: Project Review of the Dairy Association of Zambia (DAZ)’s

46

Annex 5 – Beneficiary questionnaire

Page 59: Project Review of the Dairy Association of Zambia (DAZ)’s

47

A N N E X 5 – BENEFECIARIES SURVEY

Page 60: Project Review of the Dairy Association of Zambia (DAZ)’s

48

Source: DIMS, 2020: Semi-Annual Progress Report June-December2019

Annex 6 – Results framework with reported results

Page 61: Project Review of the Dairy Association of Zambia (DAZ)’s

A N N E X 6 – R E S U L T S F R A M E W O R K W I T H R E P O R T E D R E S U L T S

49

Page 62: Project Review of the Dairy Association of Zambia (DAZ)’s

A N N E X 6 – R E S U L T S F R A M E W O R K W I T H R E P O R T E D R E S U L T S

50

Page 63: Project Review of the Dairy Association of Zambia (DAZ)’s

A N N E X 6 – R E S U L T S F R A M E W O R K W I T H R E P O R T E D R E S U L T S

51

Page 64: Project Review of the Dairy Association of Zambia (DAZ)’s

A N N E X 6 – R E S U L T S F R A M E W O R K W I T H R E P O R T E D R E S U L T S

52

Page 65: Project Review of the Dairy Association of Zambia (DAZ)’s

A N N E X 6 – R E S U L T S F R A M E W O R K W I T H R E P O R T E D R E S U L T S

53

Page 66: Project Review of the Dairy Association of Zambia (DAZ)’s

A N N E X 6 – R E S U L T S F R A M E W O R K W I T H R E P O R T E D R E S U L T S

54

Page 67: Project Review of the Dairy Association of Zambia (DAZ)’s

A N N E X 6 – R E S U L T S F R A M E W O R K W I T H R E P O R T E D R E S U L T S

55

Page 68: Project Review of the Dairy Association of Zambia (DAZ)’s

A N N E X 6 – R E S U L T S F R A M E W O R K W I T H R E P O R T E D R E S U L T S

56

Page 69: Project Review of the Dairy Association of Zambia (DAZ)’s

A N N E X 6 – R E S U L T S F R A M E W O R K W I T H R E P O R T E D R E S U L T S

57

Page 70: Project Review of the Dairy Association of Zambia (DAZ)’s

58

Page

No.

Indicator Definition How

measured/

calculated

Sources of

data

Results Comments

28

(2019)

% increase in dairy

sector contribution to the national GDP

Monetary value

(in $) of total annual national milk production as a fraction of the GDP

National milk

production multiplied by average milk price divided by GDP

Ministry of

Livestock for national milk production figures, average price obtained from milk

processors, GDP for 2018 used to calculate 2019 position

Applying this

approach and figures we obtained a result of 0.28% against that 0.34% indicated in the framework

Ideally, calculations in the results

framework should be replicable by anyone who uses the stated sources of data. This was not the case. Using 2018 GDP to calculate 2019 position is also not ideal. Some projection for 2019 could be used??

Also, this indicator is narrowly defined. Cannot measure contribution to the dairy SECTOR just by milk production. What about other services provided. This indicator may be beyond DAZ’s capacity to measure

28

(2019) No. of jobs in the dairy sector

Not clear how

these are defined.

Baseline value

was provided by Mr. Valetta, a consultant

Not clear Not clear Baseline figures unchanged in 2019 at 27,000

employees in the dairy sector

While it is expected that employment may have stagnated, it is improbable that the numbers could remain unchanged!! E.g where profit margins are declining, employers tend to downsize; where dairy farmers are

assumed to be employed in the dairy sector, their numbers do change annually, albeit marginally.

28 (2019)

Average litres of milk per cow per day

N/A Both the baseline and

N/A Here, it is questioned the

Is the DIMS project always going to rely on WeEffect as a source of data?

Annex 7 - Comments on selected indicators in DAZ DIMS result framework

Page 71: Project Review of the Dairy Association of Zambia (DAZ)’s

A N N E X 7 – C O M M E N T S O N S E L E C T E D I N D I C A T O R S I N D A Z D I M S R E S U L T F R A M E W O R K

59

for targeted small end of year inconsistency on The question still remains on how

scale dairy farmers values in the how the indicator WeEffect estimates these numbers. (2019 RF) / % 2019 RF were is stated and the

increase in average obtained from baseline figures for

milk (litres of the WeEffect this indicator in the

milk??) per cow per Results 2 RFs. In the

day for targeted Assessment Jun2018-May2019

small scale dairy for 2018 RF which is also

farmers (2018 RF) which year 1, the

coincided with percentage

DIMS end of increase is 0%

year reporting BUT in the

Jun2019-

Dec2019RF, year 1

results are set at

an average of 7.84

litres of milk equal

to an increase of -

0.04% as

compared to the

baseline

28 Average litres Here, it is M&E to check accuracy of numbers

(2019) delivered by questioned how when reporting targeted small scale the figure 33.5%

farmers daily in the last column

was obtained. In

actual fact the

increase from the

baseline of 14.28

to 21.5 litres is

7.22 litres, a 51%

increase.

29 Average increase in Number of days Figures are At the time of 280 days against a The approach used to collect data is

(2019) lactation period a cow is continuously

averages over averages

reporting, Field facilitators do

baseline of 200 interviews of a purposive sample (about 10) of farmers who have been

producing milk reported by adhoc interviews trained in each locality covered by a Field with farmers who Field Facilitator.

Facilitators. have been trained in cattle

The average should be taken over all farmers, both trained and untrained to

Page 72: Project Review of the Dairy Association of Zambia (DAZ)’s

A N N E X 7 – C O M M E N T S O N S E L E C T E D I N D I C A T O R S I N D A Z D I M S R E S U L T F R A M E W O R K

60

nutrition and herd health management

reflect the real progress against the baseline. This is likely to be lower than the indicated number

31 % reduction in time A purposive 15 % reduction in 30% cumulative reduction is recorded.

(2019) spent on recording milk deliveries

sample of 20 MCCs was chosen

time recorded by end of 2019 is the

Percentages cannot be added cumulatively as shown.

to measure this. correct position Details of the survey should be kept for Recorders were reference. These are currently not asked how much available. time they spent

recording the

transactions

compared to

manual entries

and averages

taken

P32 Proportion of 31.5% is correct Proportions cannot be added

(2019 targeted small scale position for end of cumulatively as shown

dairy farmers accessing finance

2019

Page 73: Project Review of the Dairy Association of Zambia (DAZ)’s

61

Source: DIMS Project Manager, March 2020

Operational Cost Involves - DSA, Fuel costs, Office Rentals for Field Staffs, etc

Social Costs - Gratuity at 25% of staff salary, Pension costs @ 5%

Designat

ion

DIMS

Proje

ct %

time

alloca

tion

DAZ

and

Other

Proje

ct %

time

alloca

tion

Monthly

staff

Salary

DIMS

Monthly

staff

Salary

DAZ and

Other

projects

Monthly

staff

Social

Costs

DIMS

Monthly

staff

Social

Costs

DAZ and

Other

projects

Monthly

staff

total

costs

Num

ber

of

mont

hs

assig

ned

to

DIMS

Total

Monthly

staff

expendi

tures

charged

to DIMS

Total

staff

Operatio

nal

Costs

DIMS

projects

Total

staff

Operatio

nal

Costs

DAZ and

Other

projects

1 Executive Manager

50 50 30000 30,000 10,500 10,500 81,000 19 40,500 20,012 8,000

2 Project

Manager

90 10 40000 0 14,000 0 54,000 19 54,000 30,353 4,000

3 Manager -

Finance

and Administr

ation

90 10 30000 0 10,500 0 40,500 19 40,500 431 3,000

4 Gender,S

avings

and

Credit Officer

90 10 18500 0 6,475 0 24,975 19 24,975 19,858 4,000

Annex 8 - Monthly Staff Costs

Page 74: Project Review of the Dairy Association of Zambia (DAZ)’s

A N N E X 8 – M O N T H L Y S T A F F C O S T S

62

5 Media

Liaison

Officer

90 10 18500 0 6,475 0 24,975 19 24,975 7,696 5,000

6 Business

Developm

ent Officer

90 10 18500 0 6,475 0 24,975 19 24,975 26,778 5,000

7 Dairy

Developm

ent

Officer

50 50 0 18,500 0 6,475 24,975 19 0 6,000 27,681

8 Accounts Assistant

25 75 1400 3,600 490 1,260 6,750 19 1,890 0 0

9 Policy,

M&E Officer

50 50 16500 16,500 5,775 5,775 44,550 19 22,275 34,369 18,000

10 Field

Facilitator - Choma

90 10 6,900 0 2,415 0 9,315 19 9,315 47,443 3,795

11 Field

Facilitator - Monze

90 10 6,900 0 2,415 0 9,315 19 9,315 30,761 1,538

12 Field

Facilitator - Lusaka

90 10 6,900 0 2,415 0 9,315 19 9,315 55,727 5,600

13 Field

Facilitator - Kabwe

90 10 6,900 0 2,415 0 9,315 19 9,315 41,268 3,714

14 Field

Facilitator - Ndola

90 10 6,900 0 2,415 0 9,315 19 9,315 48,716 3,410

15 Field

Facilitator - Kalulushi

20 80 0 6,900 0 2,415 9,315 19 0 47,063 2,118

Page 75: Project Review of the Dairy Association of Zambia (DAZ)’s

A N N E X 8 – M O N T H L Y S T A F F C O S T S

63

Total 207900 75500 72765 26425 382590 285

280,665 .00

416,474. 50

94,856.5 7

Page 76: Project Review of the Dairy Association of Zambia (DAZ)’s

Project Review of the Dairy Association of Zambia (DAZ)’s Digital Information Management System (DIMS) projectThe review of the DIMS project was commissioned by the Embassy of Sweden, Lusaka. The objective of the project, which is funded by Sida, is to contribute to a viable and sustainable dairy industry by developing a digitized system for transactions and improved access to information and dairy farming inputs. The review is based on information gathered through document study, field visits and interviews and meetings with beneficiaries and stakeholders. The main conclusion of the review is that while the DIMS system has been introduced, it is still in its infancy and needs to be complemented and further developed in order to become fully useful to the farmers, thereby creating incentives for its future financing. The review recommends continued support to the project.

SWEDISH INTERNATIONAL DEVELOPMENT COOPERATION AGENCY

Address: SE-105 25 Stockholm, Sweden. Office: Valhallavägen 199, StockholmTelephone: +46 (0)8-698 50 00. Telefax: +46 (0)8-20 88 64E-mail: [email protected]. Homepage: http://www.sida.se