end of project evaluation and future programming …...un joint project on strengthening the...

UN Joint Project on Strengthening the National Statistical System: End of Project Evaluation

1

End of Project Evaluation and Future Programming entry-points in the field of statistics and evidence-based policy

making

Evaluation Report

Arkadii Toritsyn, International Consultant Gheorghe Caraseni, National Consultant

July 2013


2

Acknowledgements The evaluators wish to thank all those who have contributed to this evaluation report. The evaluation team expresses their sincere gratitude to Nicola Harrington-Buhay, UN Resident Coordinator Moldova, for her strategic guidance and support of the evaluation. The evaluators would like to thank UN partners who effectively collaborated on this Joint Project, shared their impressions about the Project and brainstormed options for its continuation. UNDP was represented by Alexandru Oprunenco, UNICEF by Elena Laur, UNFPA by Viorel Gorceag, UN Women by Corneliu Eftodi and ILO by Oxana Lipcanu.

The consultants acknowledge strong commitment and support of the evaluation mission provided by the management of the Project’s National Partner — Lucia Spoilă, General Director of the National Bureau of Statistics of the Republic of Moldova, Vitalie Valcov and Oleg Cara, Deputy General Directors of the same Bureau. The evaluation team is especially grateful to Oleg Cara, playing the role of Project’s National Coordinator, who provided contextual information and answered numerous consultants’ questions as well as generously shared his ideas on potential interventions to strengthen National Statistical System in Moldova.

Special thanks are due to Aurelia Spataru, Project manager, for her extensive support, sharing all the required documentation and promptly and extensively responding to evaluators’ questions. The authors would like to thank also Roman Rosca, Project assistant, for his outstanding administrative support.

The authors are grateful to UN Coordination Team, NBS staff and experts, Government officials from central agencies and line ministries, academic institutions, think tanks, NGOs, media representatives and other partners and beneficiaries for sharing their views regarding the Project’s performance, relevance, impact, sustainability and future prospects.

Every effort has been made to ensure that the information in this report is correct. Any factual error that may appear is unintended and is the responsibility of the consultants. This report represents the views of the consultants only and does not necessarily represent the views of the UN Agencies or other organizations or persons referred to in this report.


3

ABBREVIATIONS AND ACRONYMS .............................................................................................................4

EXECUTIVE SUMMARY .................................................................................................................................5

PROJECT DESCRIPTION AND KEY PARTNERS .......................................................................................18 2.1 Background .......................................................................................................................................18 2.2 Project Theory of Change .................................................................................................................19

EVALUATION SCOPE, OBJECTIVES AND METHODOLOGY ...................................................................22 3.1 Purposes and scope of the end of Project evaluation ...................................................................22 3.2 Evaluation methodology ..................................................................................................................24 Evaluation Matrix ....................................................................................................................................26 3.3 Ethical considerations ......................................................................................................................28 3.4 Human Rights and Gender Equality considerations ......................................................................29

EVALUATION LIMITATIONS AND MITIGATION APPROACHES ...............................................................29

EVALUATION FINDINGS .............................................................................................................................30 5.1 Relevance ..........................................................................................................................................30 5.2 Effectiveness .....................................................................................................................................34 5.3 Efficiency ...........................................................................................................................................46 5.4 Impact .................................................................................................................................................47 5.5 Sustainability .....................................................................................................................................50 5.6 Human rights-based approach, equity and gender issues ...........................................................52 5.7 Partnership strategy .........................................................................................................................54

SUCCESSFUL PRACTICES AND LESSONS LEARNED ............................................................................54

CONCLUSIONS ............................................................................................................................................56

RECOMMENDATIONS .................................................................................................................................58 8.1 Recommendations for NBS ..............................................................................................................61

8.1.1. NBS should seek international partners’ funding and advisory support to introduce a coherent IT system and optimize structure and capacities of regional offices of NBS.............................................62 8.1.2 NBS may identify a limited number of administrative data collected by other bodies as “official statistics” and strengthen capacities of these bodies to produce quality and reliable data ...................63 8.1.3 NBS can explore and implement a range of measures to strengthen society’s confidence in statistics ................................................................................................................................................64 8.1.4 Establish an NBS Education and Training Center .......................................................................66

8.2 Recommendations for UN partners: ...........................................................................................67 8.2.1 Identify one or two priority areas for the next project of critical importance to sustainable human development in Moldova.......................................................................................................................67 8.2.2 Strengthen NBS capacity to monitor implementation of the UN-Republic of Moldova Partnership Framework 2013-2017 .........................................................................................................................68

8.3 Recommendations for UN partners on the next project design and implementation modalities ..................................................................................................................................................................69

ANNEXES ......................................................................................................................................................71 9.1 Questions used during semi-structured interviews that were tailored to each target audience ...................................................................................................................................................71 9.2 List of interviewed individuals and participants of focus groups ...........................................75 9.3 Exploratory questions to inform the Concept Note development ...........................................77 9.4 Literature, Project materials and other sources used ..............................................................78 9.5 Terms of References ....................................................................................................................84


4

ABBREVIATIONS AND ACRONYMS

BPfA Beijing Platform for Action

CEDAW Convention on the Elimination of All Forms of Discrimination against Women

CPA Central Public Administration

CSO Civil Society Organizations

DAMEP Division of Policy Analysis and Monitoring

EC European Commission

EU European Union

FDI Foreign Direct Investment

ILO International Labor Organization

LPA Local Public Authority

MDG Millennium Development Goals

ME Ministry of Economy

M&E Monitoring and Evaluation

MLSPF Ministry of Labour Social Protection and Family

MoH Ministry of Health

NBS National Bureau of Statistics

NDS National Development Strategy

NGO Non-Governmental Organization

NEX National Execution Modality

NHDR National Human Development Report

PSC Project Steering Committee

RM Republic of Moldova

ToR Terms of Reference

TUS Time Use Survey

UN United Nations

UNCT United Nations Country Team

UNDAF United Nations Development Assistance Framework

UNDP United Nations Development Program

UNFPA United Nations Population Found

UNESCO United Nations Educational, Scientific and Cultural Organization

UNICEF United Nations Children’s Fund

UNPF United Nations Partnership Framework

UN Women United Nations Women

WB World Bank

WHO World Health Organization


5

EXECUTIVE SUMMARY

The UN Joint Project “Strengthening the National Statistical System in Moldova” (hereinafter Project) was designed relying on the findings of the Common Country Assessment, conducted jointly by the Government of Moldova and the UN system in 2005. The Project was designed on the basis of the needs identified by the NBS to enable it to perform its basic role of coordinating the statistical system ensuring standards to statistical production, and providing relevant, accurate, timely, comparable, coherent and complete statistical data. The five UN agencies, namely UNDP, UN Women, UNFPA, UNICEF and ILO, have joined efforts to provide more strategic and coordinated interventions to the Government of Moldova through the Project aimed to strengthen data production, dissemination and use. The Project specific expected results from strengthening the national statistical system were:

Improved capacities of NBS and other line ministries, which are responsible for the production of official statistical information, to produce data of appropriate quality in a timely manner and with a necessary level of desegregation by gender, age and geographical dimensions;

Enhanced availability of statistical data through the improvement of data dissemination systems, practices and tools (availability of high-quality statistical publications addressing needs of data users, development of "easy-to-use" thematic materials for data dissemination, informing the users about the available materials);

Increased use of available data for participatory policy-making through enhancing statistical users’ literacy, increasing the capacities within the Government, CSOs and other stakeholders, as well as encouraging in-government and outsourced policy analyses on the basis of available statistics.

This evaluation took place in March-May 2013. The evaluation team spent one week in Chisinau conducting interviewees and focus groups and examining the documents not covered during the desk review. It was a formative exercise, the results of which would be used by Project stakeholders to inform development of a new Project in the area of statistics and evidence-based policy-making. The specific objectives of the evaluation include:

Provide a full and independent assessment of the achievements and results, failures and constraints of the Project, as well as an analysis of its performance, management, impact, relevance and sustainability of the interventions;

Generate lessons learned and good practices from each of respective outputs established in the Project since its inception in 2007 to inform current and future programming at the country level;

Assess whether the results achieved are relevant for the current statistics related environment in the country and provide solid basis for UNDP and other partner UN agencies for future programming;

Provide sharp and forward-looking recommendations that can guide UNDP, partner UN agencies and other players in developing effective, realistic and evidence-based strategies.


6

A large set of different and complementary evidence were collected and analyzed by utilizing both qualitative and quantitative methods that included:

Desk review of relevant documents;

Consultations with UN management and Project team;

Semi-structured interviews with pre-determined sets of questions with a large number of Project partners and beneficiaries;

Four focus group of Project beneficiaries and partners;

Informal conversational interviews;

Validation of preliminary findings and recommendations with the UN Project partners;

A wrap up session with the UN Resident Coordinator. The consultants reconstructed a Project Theory of Changes and developed the Evaluation Matrix.

Key findings Relevance The Project design is relevant vis-à-vis the overall Project goals. In Spring 2013 when this evaluation is conducted, the Project remains highly relevant to realities of reforms of statistics and evidence-based policy making in Moldova. The Project correctly identified and addressed fundamental limitations in the areas of statistics and evidence-based policymaking. It has addressed key limitations of the national statistical system:

Relatively underdeveloped NBS data collection and dissemination systems that faced difficulties to provide timely, quality statistics to support policy development, implementation and monitoring processes.

Statistical indicators and methodologies were only partially aligned with the Eurostat and other international standards.

Primary statistical data were often difficult to obtain and use and the users did not have an effective feedback mechanism with NBS.

Many statistical indicators were not disaggregated by sex, age, area, social group, etc. or, if disaggregated, were not effectively disseminated in user-friendly format.

The users had limited awareness of the importance of statistical information and insufficient skills to use data to inform policy development in line ministries, interpret social and economic events and other purposes.

The Project design and its objectives remain highly relevant vis-à-vis national policies and strategies. The Project strengthened the Government’s capacity to report on its progress in meeting a range of international commitments. The political and economic realities of Moldova have changed through the course of Project implementation and the Project adjusted its interventions to support new Government’s priorities. The evaluation team finds that the Steering Committee established by the Project is an effective mechanism to maintain its continuous relevance to NBS’ and partners’ needs. The Steering Committee has become an efficient platform to coordinate the donor support provided to NBS/statistical sector as during the Committee meetings the UN agencies kept each other


7

informed on their activities in the area of statistics that targeted NBS, line ministries and relevant agencies that helped to avoid duplication of their efforts and achieve synergy. Through annual review of Project priorities, the Steering Committee identified and revised Project’s priorities and activities. High Project relevance was acknowledged by the interviewees and participants of all focus groups. The Project was responsive to partners’ needs. Effectiveness The Project strengthened data collection, storage, compilation, dissemination and statistical analysis, and improved gender, social, labour, demographic and other statistics within the NBS. It implemented a range of activities to improve statistics use by policy-makers and other key users, as well as made data access and dissemination consistent with the international best practices. The Project design was broad enough to facilitate a variety of activities and provided the management with a significant level of flexibility in responding to partners’ needs. NBS’s capacities have been strengthened in such diverse areas as demographic, e-development, environmental statistics and sample-based surveys data collection processes. Statistics to support effective decentralization were enhanced. In the area of labour statistics, Project supported diverse activities involving ILO experts that significantly improved quality of labour statistics. NBS statistical toolkit for the survey on enterprises was harmonized with the requirements of the National Accounting Standards. Support in preparations for the Population and Housing Census 2014 was provided through advocacy and methodological support. Quality management in statistics has been improved through training of regional statistical offices staff. The NBS website content and user-friendliness were significantly improved due to the Project’s support that was appreciated by multiple users interviewed through the course of evaluation. NBS management and staff skills and capacities were strengthened through targeted training and international exposure. Statistical methodologies in many areas were aligned with the EU and international standards that facilitates cross-country comparisons and supports Moldova’s EU integration processes. National definitions and calculation methods for 5 MDGs indicators, for example, were updated and harmonized with the international standards and disseminated due to the Project support. The Project provided expertise and support to ensure compliance of statistical methodology and toolkit for Time Use Survey, Social Exclusion Survey, Integrated Statistical Survey on Enterprises, Research & Development statistics with the Eurostat and other international standards and good practices. Concepts, methodologies and questionnaires for a number of statistical surveys (e.g., labour costs, labour time & wages, annual structural statistics, work accidents) were revised to ensure compliance with the national legal and regulatory framework and needs of key stakeholders, as well as the EU/international standards and practices. Capacities of NBS in production of metadata have been strengthened. Metadata for 28 statistical fields and surveys, and 28 core statistical indicators that derived from national MDGs were produced and made publicly available.1 The evaluation team reviewed the metadata and sectoral

1 See http://www.statistica.md/pageview.php?l=ro&idc=430& and http://www.statistica.md/pageview.php?l=ro&idc=433&

http://www.statistica.md/pageview.php?l=ro&idc=430&



8

data available on the NBS website and confirm that the metadata supported by the Project are of high quality, user-friendly and provide rich information. The interviewees from line ministries and central agencies were highly satisfied with the quality and accessibility of the metadata. Social statistics production, dissemination and use have been significantly improved. Social statistics indicators have been significantly expanded. A number of statistical publications and thematic materials (leaflets, synthesis, analytical papers, etc.) on social topics were supported and produced in user-friendly and easy-to-use format and widely disseminated. The Project published the Approaches to Social Exclusion in the Republic of Moldova that provides the tools, including disaggregated statistics, to monitor social inclusion and identify policy and programs priorities. The evaluation team acknowledges high quality and user friendliness of all of the Project publications. Gender was mainstreamed into multiple areas of statistics, capacities of NBS and data users were strengthened in gender statistics and high quality analytical products were produced. A set of harmonized gender sensitive indicators, in the context of the MDGs and National Program on Ensuring the Gender Equality, was developed through multi-stakeholder consultations that involved more than 100 partners. The process of harmonization of gender-sensitive indicators was identified by experts as a successful practice and was replicated in several countries in the CIS. Capacities of the NBS to conduct gender-specific surveys have been strengthened to produce multi-dimension disaggregated data. The survey ‘Violence Against Women in Family’ was designed through consultations with various partners. The data on violence against women were collected, disaggregated, analyzed and used for advocacy and policy making. Gender was mainstreamed into numerous surveys undertaken by the NBS. Analytical reports on “Women and Men in the Republic of Moldova” and “Working children in the republic of Moldova” are just a few examples. Pocket books “Women and Men in the Republic of Moldova”, “Gender Statistics User Guide”, “Population: Definitions and Indicators Guide” were published. Capacities of statistics users have been strengthened. The Project conducted numerous training and workshops for participants from line ministries, including gender focal points, on practical skills in using and interpretation of disaggregated sector sub-sets of indicators. A number of public awareness events that demonstrated the importance of statistics to decision-making were conducted. The events targeted representatives of NBS, respective line ministries, representatives of local NGOs, think-tanks and academia, international organizations. A number of research studies that relied on official statistics were conducted by independent think-tanks (foreign direct investments, rural labour force, public health status, impact of prices on the well-being) were supported and publicly released. The studies demonstrated the value of quality statistics and contributed to development of evidence-based decision-making culture in respective ministries. State University of Moldova, department of journalism and communication science, was supported in developing courses with special focus on statistical notions and process, documentation, interpretation and use of statistical data in the journalistic/communications activity/profession. The courses will be launched in 2013. The consultants made the following general observations regarding the Project effectiveness:

Socio-economic and political conditions in the country were unstable during the Project implementation that negatively affected its effectiveness in the field of building users’ capacities.


9

The mere number of activities and results achieved by the Project demonstrates that it gained respect of key partners that entrusted the Project with significant responsibilities.

The evaluation team found that the Project adopted a correct approach of providing support to all elements of the national statistical system, going beyond NBS. Improvements were needed in all parts of the system, including line ministries and other producers and users of statistics. As a result, the Project components well interact and complement each other, especially in the fields of gender and social statistics.

The Project’s effectiveness was highly evaluated by NBS management and staff and many other partners and statistics’ users. The evaluation team heard predominantly very positive feedback about the Project.

Some Project activities increased line ministries’ capacities to use statistics in policy development, implementation and monitoring, but they frequently lack the necessary knowledge and skills to use statistics in the policy cycle, including such areas as conducting research, developing policy options, conducting ex-ante policy assessments as well as estimating costs of new policy initiatives. The expertise of local governments in using statistics and other data in their decision making is weak and insufficient to support decentralization pursued by the Government.

One of the main vehicles of supporting NBS and other partners used by the Project was hiring short-term consultants to meet specific time bound deliverables (e.g., develop training manuals, review statistical methodologies and processes, produce analytical reports on the basis of statistical data). A proper mix of national and international consultants was maintained. The evaluation team found that the approach of hiring experts from new EU member states was effective in terms of transforming the knowledge and expertise to the counterpart staff. The contributions of many international and national consultants have been highly commended and the overall services of most consultants hired were of excellent quality.

The Project provided extensive multiple capacity building interventions for diverse beneficiaries. The Project does not have however a comprehensive system of monitoring the actual relevance, usefulness and impact of these interventions.

Partially due to the Project interventions, users’ satisfaction with NBS’ products have improved. According to two user satisfaction surveys, NBS’ independence and expertise is well acknowledged by the public. Among all government services the NBS enjoys a high level of trust.

The Project managed to improve significantly NBS’ social statistics field, including introduction of new and disaggregated indicators aligned with the international standards and producing solid analysis that supported the actual policy processes. Transparent, reliable, timely, and comparable estimates of poverty, inequality and other social inclusion indicators are essential for a wide range of policy and program decisions.

The data and metadata bases and development of electronic data dissemination tools accessible on-line to the general public ensured transparency and easy access to data and built sustained demand for quality, timely and accessible statistics.

Other areas where the project effectiveness can be improved are: better alignment of data produced by NBS and line ministries, improved timeliness of data dissemination, provision of more disaggregated data, and better institutionalized collaboration of NBS and line ministries.


10

The Project team consists only of two individuals (a Project manager and an assistant) that puts a significant pressure on the manager to develop and oversee multiple activities.

Efficiency The Project exceeded its objectives within established timeframes and costs and has demonstrated good efficiency. The Project strived to meet its expected outputs by looking for cost-efficiencies in Project operations and utilizing prudent business practices. The Project contains elements of soft assistance (e.g., reviews of statistical methodologies, trainings) and these soft elements are embedded into various Project components that make the application of conventional efficiency indicators to these areas not feasible. The evaluation team does not believe that the costs of Project deliverables can be lowered while still achieving Project objectives. The alternative delivery methods such as hiring permanent statistics experts placed in the NBS would be more costly and not responsive to evolving needs of the NBS and other partners. The evaluation team found financial documentation easy to understand and it is well kept, given the extent of challenges associated with managing a Project with pooled and parallel funding.2 The following steps were undertaken by the Project management to ensure quality and cost-effectiveness of the process of transforming inputs into outputs and outcomes:

UN Project office was located in NBS that did not require renting of Project premises and resulted in substantial savings over 7 years.

Partners applying for grants to produce statistics-based analytical products went through a rigorous process of selection by a (multi-ministerial) representative committee and were required to make their own contribution to their project costs to qualify.

Open, transparent and value-for-money UNDP procurement processes were utilized for all procurement and human resources processes.

Internal UNDP, ILO, UNICEF, WHO, UNESCO and other international organizations’ experts and consultants with significant expertise in statistics and other relevant fields were often mobilised with no remuneration being paid.

The experts with significant knowledge of comparable jurisdictions (e.g., Croatia, Poland, Romania) and relevant language skills (Romania) were mobilised that resulted in cost savings and more effective advice based on the recent experiences of EU accession processes.

All consultants, national and international, were involved on short-term, as needed basis instead of long-term contracts that generated significant cost-savings and helped to target very specific partners’ needs.

Local experts were engaged whenever possible instead of international consultants. The Project demonstrated a high degree of flexibility in adapting to the changing environment. The Project quickly responded to changing Government priorities and implemented a range of activities to improve the quality and availability of regional statistics to support the Government’s decentralization and regional development strategic documents.

2 For instance, funds spent by UNFPA, ILO, UNICEF.


11

Impact The Project activities contributed to systemic changes in the field of statistics and evidence-based policy making in Moldova. The following key areas of impact have been identified:

Numerous statistical indicators designed with the Project support are produced on regular basis and most likely will continue to be produced and disseminated by the NBS after the Project completion that demonstrates NBS’ confidence about data methodologies and its ownership.

Statistical methodologies in many areas were aligned with the EU and international standards that facilitate cross-country comparisons and support Moldova’s EU integration processes.

Capabilities of NBS in data collection and dissemination and responsiveness to users’ needs have significantly improved. The accuracy, quality, and in some instances timeliness of statistics have improved. A range of easily accessible statistical products has increased.

NBS’ and Government’s capacity to monitor Moldova’s progress toward the Millennium Development Goals has significantly improved so that the country’s progress towards MDGs is assessed relying on international methodologies and the data produced are reliable.

Some foundational elements of the evidence-based decision-making culture were established. The users and data producers themselves started to appreciate more the value of quality, timely and disaggregated statistics that generated demand for better statistics. Positive changes occurred in culture and practice in statistical and other evidence use by line ministries.

The users’ levels of trust in statistics have improved significantly and this can be to a large extent attributed to Project’s interventions.

The NBS website became an essential source of statistics for increasing numbers of users.

Statistics are more extensively reflected in Government’s Strategies, Programmes and policies, in particular in social and gender fields that were specifically targeted by the Project.

Although a number of high quality analytical publications were produced by the Project (e.g., social inclusion, gender, working children, etc.) and these publications were widely disseminated, the actual impact of these activities remains under-examined.

Users’ satisfaction with the available NBS’ statistical products has improved. A range of data users has increased and includes more NGOs, academic institutions, journalists and students, which stared to use official statistical data in their activities.

The capacities of partners that benefitted from the Project’s training opportunities have increased.

Sustainability The evaluation team found that there is a substantive level of ownership of the Project within the NBS, at both management and staff levels who enhanced their skills and confidence to carry the reform process forward. The NBS management and staff were very clear in identifying their strengths and needs and were open to exploring new statistical tools, processes, methodologies and adopt successful international experiences. Sustainability of the Project, however, is tightly


12

connected with the degree of political commitment to public administration reforms and recognition of the importance of statistics in policy making and monitoring. The main findings regarding Project sustainability include:

Numerous statistical indicators and methodologies that were developed by the Project are deeply institutionalized in the NBS. They are routinely produced and most likely will continue to be produced by the NBS after the Project completion that demonstrates NBS’ confidence about data methodologies and ownership.

Due to the Project, statistical methodologies in many areas were aligned with the EU and international standards that facilitates cross-country comparisons and support Moldova’s EU integration processes.

The Project strengthened NBS capacities in the area of social and gender statistics. The evaluation team does not believe that the extensive social and gender surveys supported by the Project will be regularly produced by NBS without external support as it requires significant budget resources. Such ad hoc surveys may be continued by NBS if they support the identified policy priorities.

NBS developed a comprehensive list of 41 needs for future development of national statistical system, while Central Public Administration institutions formulated 23 proposals. They partially reflect Project’s successes and demonstrate that partners understand the importance of statistics for data evidence-based policy-making and are willing to collaborate with development partners within the new RM-UN Partnership Framework 2013-2017.

The Project capacity building activities targeting both data producers and users made a significant impact. For instance, the training course on statistical literacy for students at journalism and communications department of the State University has been approved by the University’s Senate as part of curricula. The Project also developed for NBS staff working in the field – interviewers and observers curricula consisting of 3 modules that are used by NBS. These capacity building capacities may not be sustainable in the long run as they did not become institutionalized within NBS or other key Government policy institutions. With the risk of staff rotation/attrition, some of the Project’s achievements in capacity building will be unsustainable, if not institutionalized.

There are a number of external factors beyond the Project control that positively contribute to its sustainability:

The Government EU accession plans will have to be supported by continuous improvement of statistics and their alignment with the EU standards.

The UN-Moldova Partnership Agreement identifies a number of reforms in public administration and confirms commitment of all partners to establish a modernized public administration system capable to effectively and efficiently develop, budget, implement and monitor evidence based policies in support of the country’s national priorities and European integration objectives.

There are a number of factors beyond the Project control that may negatively affect its sustainability:

Political uncertainties that may undermine Government commitment to EU integration and promoting evidence-based decision-making practices.


13

The NBS’ capacity to lead reforms of the national statistical system has to be continuously strengthened.

Global economic crisis and lack of domestic structural and economic reforms may negatively affect the economic performance of Moldova and result in declining NBS budgets that will have a detrimental effect on Project sustainability.

Partnership strategy

The Project’s success was to a large extent determined by the effectiveness of UN partnership strategy. The project represents one of the few Joint UN initiatives in the country and it demonstrated usefulness and high impact of such interventions based on synergy of UN partners. Each UN agency involved into the Project has its own areas of expertise and responsibility. The evaluation team found that effective partnership was ensured at both programmatic and funding levels where the roles of key partners were clearly delineated.

Each of UN partners relied on their extensive international networks of expertise.

As the evaluation team acknowledged in the relevance section, the Steering Committee established by the Project is an effective partnership mechanism linking the NBS, statistics users and international partners. Through annual review of Project priorities, the Steering Committee identified and revised Project’s priorities and activities that should be implemented by each UN system partners. The national partners are highly satisfied with the Project partnership strategies and they appreciate that their voices are heard and interests addressed. The UN Joint Project developed a comprehensive and well functioning coordination mechanism that works for all partners involved and the evaluation team acknowledges that the Project manager played an instrumental role in success of these coordination efforts. As there are a few partners supporting NBS such as the IMF, Statistics Norway, the Project management ensured proper co-ordination of activities with other diverse partners to encourage synergy and avoid overlaps. The evaluation team identifies the following successful practices in Project implementation:

In terms of programming, the most valuable lesson learned, that should also be viewed as a good practice example, is a strong partnership of all UN agencies that was critical to its success. The comprehensive nature of the joint Project that focuses on such diverse areas as institutional building, improving data collection, processing, use, capacity building, supporting the incorporation of data into the policy cycle and multiple beneficiaries could not effectively implemented by one UN agency. The Project adopted one UN approach and benefitted from UN agencies synergy where each UN partner contributed its expertise. It is an eloquent example when multi-stakeholder open consultative processes contribute to the practical application of good governance principles of openness and transparency, and participatory decision-making. Such processes can and should be replicated in similar initiatives in Moldova and elsewhere.

The Project was correct in addressing all key stages of the statistics process: development of indicators, enhancements of data collection and production processes, installation of quality


14

control mechanisms, data production in user-friendly format and analysis of statistics and significant users’ capacity building to utilize statistics in policy development and monitoring.

The Project addressed both supply and demand for statistics at the same time. It takes time to build demand for high-quality data that can be stimulated by training, producer-user dialogue, and effective data dissemination.

The implementation of specialized surveys and ad hoc modules to existing surveys to explore social trends and other areas strengthened capacities among NBS to produce quality statistics in line with EU and international standards and learn new methodologies. Although these exercises could be seen as an additional burden on NBS’s staff, they demonstrated the abilities of NBS to produce high quality statistics and built staff confidence. In addition, the users realized that NBS could produce diverse statistics to inform their policy work and that they need to develop skills to use the data appropriately.

In terms of management, Project implementation has shown that working in uncertain political circumstances requires significant flexibility, adjustments to implementation plans to ensure both start-up and continuity of Project activities in conditions of political stale-mate. Therefore, the identified political risks in the Project documents are never to be taken lightly, as they represent the biggest possible challenge to successful Project implementation. In such circumstances, Project subject knowledgeable, creative and dedicated Project manager with partnership building skills are some of the keys to success, as proved by this particular Project.

In terms of efficiency, the fact that the Project Team was located on the premises of the NBS, reduced overheads, enabled permanent “face to face” contact, direct communication and continuous support to the key national counterpart at any needed situation.

The Project impact and sustainability was ensured through adopting of effective institution and capacity building interventions that built solid foundations for NBS to lead the reforms of the statistical system.

Lessons learned:

Fragmented and uncoordinated capacity building activities do not result in sustainable improvements in statistical production, dissemination and use. A holistic approach to reforms is necessary as the improvements are needed in all parts of the system, including line ministries and other producers and users of statistics.

Reform of statistics cycle is a time consuming process that requires continuous commitment of multiple stakeholders and a strong political will. The reforms of statistics should be linked to promotion of evidence-based decision making culture and should have strong support of the central public administration.

NBS has the necessary authority under the current legislation to strengthen capacities of line ministries in administrative data production and use, but central public administration support is necessary to ensure that this function is effectively performed as required.

The ownership by the NBS and close coordination of all stakeholders are critical to Project success. The improvements of the national statistical system should have a clear focus on users. Users should be clearly identified and supported through diverse awareness raising and capacity building interventions to demonstrate through practical examples the benefits of quality statistics and how to use the data. Although the analytical work and supports were undertaken for the benefit of data users (e.g., how to read and use statistical data), it is insufficient to conduct seminars, roundtables and other ad-hoc activities to support them. The


15

interventions should be systemic and comprehensive, with some practical hands-on exercises tailored to users’ data needs.

The biggest challenge of any interventions in the area of statistics is ensuring sustainability of reform efforts. It is not sufficient to ensure a broad partners’ buy in of reforms and strengthening capacity of key partners is needed to ensure sustainability.

The Project’s Log Frame should be regularly updated to reflect the changing Project circumstances. The main intended outputs, activities, results and objectively verifiable indicators should be adequately defined, appropriate and stated in measurable terms.

Recommendations set forth in the evaluation report suggest possible strategic and follow-up activities for UN, NBS and other stakeholders from the governmental officials, line ministries, mass media, think tanks and civil society organizations in order to increase sustainability prospects of project achieved results, or/and to enhance their effectiveness. The following guiding principles were adopted in developing recommendations for a new UN Joint Project in the area of statistics and evidence-based policy making:

Policy and regulatory instruments of promoting use of evidence in decision-making are unlikely to work in unstable political climate of Moldova. In light of these political fragile realities, it is advisable to focus the next joint project on “soft” instruments of promoting evidence-based decision making, such as: capacity building mainly through trainings and consultancy of multiple and various statistics users, improving of quality of administrative data and alignment of methodologies used by NBS and line ministries to EU standards, and expanding a range of data dissemination instruments. In this context, it is advisable to position the next project within the NBS that is less susceptible to changes in the political leadership.

The Government’s commitment to EU accession is a strong key driving force and a leverage point to strengthen statistics and promote evidence-based policy making. In light of pro-EU orientation of Moldova, the next Joint UN project in the areas of statistics should be well coordinated with the EC Delegation.

The EC Commission is interested in launching a major project in the area of regional development that will have a significant regional statistics component. In this context, it is advisable to seek complementarity for a next UN Joint project.

The overarching goals of the next project in the area of statistics should be to contribute to creating an appropriate environment of evidence based policy making conducive to alleviating social exclusion and promoting sustainable human development in Moldova. Statistics should not be produced solely to capture Moldova’s progress with regard to meeting its international commitments, but should be deeply embedded into a policy cycle.

It is necessary to maintain a systemic approach to statistical capacity building and maintain collaboration among the relevant UN Agencies and other development partners and bilateral donors and create necessary synergy. The supports should not be based on ad-hoc requests for specific statistics and a strategic approach addressing both capacity needs of the NBS and Government policy objectives should be pursued.

The following recommendations regarding key priorities for a new project in the area of statistics and evidence-based policymaking should be seen as an agenda of possible topics for discussions between UN Agencies, NBS and other relevant Government institutions, partners


16

and donors. The needs, demands and expectations from users are pressuring NBS to provide more source information, better access to disaggregated statistics and tools, and more analysis. Recommendations for NBS

Seek international partners’ funding and advisory support to introduce a coherent IT system and optimize structure and capacities of regional offices of NBS. The introduction of a comprehensive IT infrastructure based on modern communication and information technologies for data collection, validation and processing will improve cost-effectiveness of data production, reliability and timeliness of statistical data. Equipment installation and software development should be accompanied by training of the regional NBS staff.

Identify a limited number of administrative data collected by other bodies as “official statistics” and strengthen capacities of these bodies to produce quality and reliable data. Under the Law on Official Statistics, NBS is a coordinator of the Statistical System in Moldova. NBS prepares the programmes of statistical works, develops and decides on the methodology used for the production of official statistics. Line ministries and agencies collect diverse administrative data and it is beneficial to clearly identify those important data elements that could be considered as “official statistics” data. The concepts, definitions, and methods of data collection for these specific indicators will be defined jointly by NBS and respective line ministries/agencies. The identified administrative data will be checked by NBS experts and considered as statistical data. It will also help to avoid duplication of responsibilities for data collection between NBS and line ministries/agencies and ensure cost-efficiencies.

Implement a range of measures to strengthen society’s confidence in statistics. A reliable and trusted statistical system is the key in recreating trust in the state and establishing an accountability link between the state, citizens, and private sector. Both NBS and its partners have to strengthen public trust in statistics by addressing both supply and demand simultaneously. The data provided by NBS should be relevant, timely and easy to interpret and use. The role of NBS should gradually change from the basic tasks of collecting statistics and disseminating it to assisting users to make appropriate use of the data. This means providing more explanations and interpretation of identified key trends. The statistical information should be offered in different formats and appeal to such diverse audiences as academics, students, government officials, parliamentarians, public servants, media and NGOs and be easy to understand and use. Particular focus can be made on decision makers and key opinion leaders who are in position to influence and lead broad societal changes.

Establish an NBS Education and Training Center. It is insufficient to strengthen capacities of data producers, to ensure that statistics positively contribute to human development in Moldova; the focus should be as broad as possible and target all actors of the system, including other data producers, respondents, data compilers, and data users. The evaluators are confident that the NBS has sufficient capacities and is well positioned to provide training and capacity building supports to these diverse audiences. It is beneficial to establish NBS Education and Training Center that will perform at least three core functions: addressing internal NBS capacity building and HR development needs, focusing on strengthening training policies, programs and capacity, especially of NBS’ regional units; support of diverse users, such as line ministries and LPA in understanding, interpreting and applying the data for policy development, monitoring and analysis; support universities


17

and colleges in integrating topics on how to read and use statistical information into relevant courses and/or developing stand alone courses on data use. In pursuing this objective, the NBS can rely on findings of the UN Joint Project’s feasibility study on the optimal scenario for the creation of such a Training Unit that was conducted in 2011.

Recommendations for UN partners:

Identify one or two priority areas to sustainable human development in Moldova. The project has a long experience in statistical capacity building, but in the evaluators` opinion a more systematic and focused approach has to be adopted for the next project. The partners have to prioritize areas that would have the most significant impact on human development in Moldova and reflect key Government priorities, international commitments and mandates of respective UN Agencies. One potential area worth exploring is sustainable human development. What is the impact of environmental changes on people’s lives and livelihoods? Which social groups and communities are most vulnerable to environmental changes? How to design and implement policies that would address both poverty reduction and social exclusion and environmental changes? Another area worth exploring is social inclusion. The project may reinforce its social exclusion study and reflect the current status of socially excluded groups. It may inform development of National Action Plan (NAP) against poverty and social exclusion that could be aligned in principle with the common EU guidelines in the area of social exclusion. Such areas as income poverty, long-term unemployment, health and lifelong learning and others could be covered in depth.

The following sequence of steps can be undertaken by UN partners:

Identify the most pressing data needs relying on the information available from UN partners, Government, NGOs through the lens of UNPF, Government priorities and capacity/willingness to incorporate in policy cycle;

Create sectoral groups with NBS involvement to improve quality of administrative data and ensure that the data collected under the UNPF are used in the policy cycle;

Conduct roundtables and other forms of consultations with stakeholders to explore how new/improved data collected for the UNPF could be used by a broad range of stakeholders.

In terms of Project design, it is recommended to strengthen the focus of the next project on a narrow set of outcomes and clearly identify expected results in the project document. It is beneficial for the next project to update annually or more frequently, as necessary, the Log Frame that could help to make the activities, performance measures, outputs and outcomes more current, realistic, sustainable and better focused, as well as better clarify the linkages between the inputs, outputs, means of verification, expected outcomes and ideally impact.

PROJECT DESCRIPTION AND KEY PARTNERS

2.1 Background The UN Joint Project “Strengthening the National Statistical System in Moldova” (hereinafter Project) was designed relying on the findings of the Common Country Assessment, conducted


18

jointly by the Government of Moldova and the UN system in 2005. It found that Moldova made certain progress in tackling poverty and promoting and monitoring human rights. The data collection and data dissemination systems, however, were weak and not in line with international standards, while data analysis was far from desirable and could not inform effective policymaking.

Under the provisions of the Law on Official Statistics in the Republic of Moldova,3 the National Statistical System is composed of a number of bodies of official statistics. They are the National Bureau of Statistics (NBS) and its territorial subdivisions, the statistical departments within the specialised bodies of central and local public administration and within the National Bank of Moldova.

The NBS is subordinate to the Government for the management and co-ordination of statistical activity. Its objects are the collection, processing, analysis, dissemination and building of official statistical data series with a view to substantiating and evaluating economic and social policies, the decisions of the Government and of economic units, informing the public, producing scientific researches, sending data to government bodies in compliance with the commitments made, as well as to other users, both internal and external. The NBS plays an important role in the implementation of the national strategic programmes and plans (including national development strategies), being the core provider of the official statistical data in the country and outside its boundaries. The NBS undertakes major statistical operations such as the population census, and household surveys; and produces price, labour and other statistics. The line ministries collect administrative data in their own areas of operation.

When the Project was designed in 2007, it was clear that NBS established core foundations of statistical system serving the needs of the market economy and democratic society. The global assessment of the official statistics system in the Republic of Moldova conducted in 2006-2007 concluded that the national statistical system was reasonably developed, adhered to the Fundamental Principles of Official Statistics and in many domains fully or nearly fully complied with EU requirements. Multiple donors provided assistance to the NBS for improving the statistical system, including but not limited to the World Bank, EU/TACIS, IMF, DFID, SIDA, EU/Food Security Program and UNDP. These supports helped NBS to strengthen its capacities, but the Moldova’s statistical system still suffered from inconsistent methodologies, multiplicity of data sources, weak capacity of data providers and data users, inadequate resources and lack of continuous training and capacity building for all partners involved. Most of statistical indicators were still not disaggregated by sex, age, and geographical territory; and even existing disaggregated indicators were often not disseminated and could not be easily accessible. As a result, decision makers and the public had only limited information to analyze current situation, develop policies, monitor their implementation and evaluate the impact, as well as to assess to what extend Moldova is fulfilling its international obligations. Limited statistical capacity, for instance, was a major obstacle to measuring progress of Moldova towards the Millennium Development Goals (MDGs).

The transitional processes have focused national and international attention on the quality of statistical data, and have sharply highlighted the urgent need to improve the capacity of the national statistical system. Previous development partners’ initiatives of supporting the NBS

3Law No 412/9.12.2004 published in the Official Journal of 7.01.2005


19

capacity have been patchy, insufficiently coordinated and partly driven by availability of external resources, mainly representing stand alone initiatives. Therefore, there was a clear need to develop a long term, sustainable, comprehensive, holistic, partnership based and coherent and realistic Project strengthening all key elements of the national statistical system.

The Project was designed on the basis of the needs identified by the NBS to enable it to perform its basic role of coordinating the statistical system ensuring standards to statistical production, and providing relevant, accurate, timely, comparable, coherent and complete statistical data. The five UN agencies, namely UNDP, UN Women, UNFPA, UNICEF and ILO4, have joined efforts to provide more strategic and coordinated interventions to the Government of Moldova through the Project aimed to strengthen data production, dissemination and use.5 The UNDP played the leading role taking the overall coordination and turning into the main driving force. The Project was extended in 2010 for 2011-2012 years and then in 2012 extended until June 2013.6

2.2 Project Theory of Change The main objective of the Project is to improve efficiency and effectiveness of the national statistical system to provide relevant, timely and reliable disaggregated data in line with internationally accepted methodology and best practices. The Project’s goals were to upgrade the conceptual, methodological and analytical skills of the NBS and other data producer and user agencies of the country. The Project is aimed at strengthening the national statistical system through improvement of data collection, dissemination and utilization of statistical information, with particular focus on national needs and overall conformity of official statistics with international standards. Project includes three components with specific focus on strengthening national capacities in accurate production of qualitative statistical data, their timely dissemination and proper use. The Project specific expected results from strengthening the national statistical system were:

Improved capacities of NBS and other line ministries, which are responsible for the production of official statistical information, to produce data of appropriate quality in a timely manner and with a necessary level of desegregation by gender, age and geographical dimensions;

4A pooled and parallel fund management modality was implemented by the Project’s partners. See, for instance, Memorandum of Understanding between UNIFEM/UN Women and UNDP as of 2007 and 2010, as well as Memorandum of Understanding between the Participating UN Organizations and the Managing Agent - for pooled fund management. 5 Government of the Republic of Moldova, United Nations Development Programme, United Nations Population Fund, United Nations Development Fund for Women, Joint Project Document, Strengthening the National Statistical System, 2007 and Government of the Republic of Moldova, United Nations Development Programme, United Nations Development Fund for Women (UNIFEM – part of UN Women), United Nations Population Fund, United Nations Children’s Fund, International Labour Organization, Amendment to the Joint Project Document, Strengthening the National Statistical System of the Republic of Moldova, period of extension 2011-2012. 6 Minutes of the Steering Committee meeting, 24 November 2010, Letter of the UN Resident Coordinator in Moldova to Mr. Victor Bodiu, State Minister of the Republic of Moldova. 12 January 2012, Minutes of the Steering Committee meeting , 24 October 2012.


20



This final project evaluation covers the whole Project for its entire duration from mid 2007. The evaluation team found design of the Project logical, with well-linked elements and effective division of responsibilities among key partners and stakeholders. The Project addresses key limitations of the national statistical system in Moldova.

The evaluation team developed a simplified Theory of Change based on the Project Log Frame, reports and other documentation. It reflects the underlying rationales and assumptions that define the relationships or chain of results that lead Project strategies and activities to intended results. The Graph 1 below presents a chain describing how the Project has worked and established plausible associations between the Project interventions and the results sought. Graph 1. Simplified Project Theory of Change


21

The Project is implemented using both, the National Execution Modality (NEX) and Direct Execution Modality (DEX). A National Project Coordinator was appointed by the NBS. The NBS also provided the Project team office space free of charge, representing the Government contribution towards the Project implementation.

The Project established a Project Steering Committee (PSC) that includes a National Project Coordinator, representatives of other state institutions and donor agencies. The PSC was meeting regularly to examine and discuss the Project work plans and reports, and provide recommendations on the Project activities. Although the NBS was a main partner and beneficiary of UN Joint Project, the Project involved and benefitted a very wide range of partners that include key statistics users and producers of administrative data. Some of these partners are listed below:

State Chancellery – promotion of evidence-informed decision making and use of data in decentralization, improvement of national definitions and concepts for MDGs

Ministry of Economy –use of statistics for analysis of direct foreign investments, support to production of Small Area Deprivation Index

Strengthened National Statistical System

Improve Data Availability

Improved statistical infrastructure,

methodologies and indicators

Extensive NBS staff trainings

Improved data dissemination tools,

metadata

Improve Data Dissemination

Improve access of data users to disaggregated

data

Publications and website with easy to use

statistics

Targeted awareness campaigns

Improve Data Use

Extensive trainining for diverse groups of users

Direct support of line ministries with data use for policy development

and monitoring


22

Ministry of Environment - improvement of national definitions and concepts for MDG 7 related to environment

Ministry of Education –improvements in estimation of indicators MDG 2 as they relate to access to general education, administrative data production, use of statistics in decision making

Ministry of Health – improvements in estimation of indicators MDGs 4-5, use of statistics in decision making through analysis of health statistics, administrative data production

Ministry of Labour, Family and Social Protection – mainstreaming gender into statistics, labour market and living standards statistics, social inclusion, us of gender statistics in policy making and analysis, accidents at work and statistics on disability

Local Governments – data providing and use of statistics in decision making

Numerous academic institutions and NGOs - use of statistics, production of analytical reports based on statistics

State University of Moldova – development and institutionalization of a statistical literacy course for future journalists and communication specialists

Media - advocacy and media outreach campaigns, public roundtable discussions, building of capacities to use statistics.

EVALUATION SCOPE, OBJECTIVES AND METHODOLOGY

3.1 Purposes and scope of the end of Project evaluation This evaluation took place in March - May 2013 with the 5 days filed mission in Chisinau in March 2013. As mentioned, the evaluation team spent one week in Chisinau conducting interviewees and focus groups with a wide range of stakeholders7 and examining the documents not covered during the desk review. It was a formative exercise, the results of which would be used by Project stakeholders to inform development of a new Project in the area of statistics and evidence-based policy-making. This is an end of Project evaluation.

The specific objectives of the evaluation include: 1. Provide a full and independent assessment of the achievements and results, failures and

constraints of the Project, as well as an analysis of its performance, management, impact, relevance and sustainability of the interventions;

2. Generate lessons learned and good practices from each of respective outputs established in the Project since its inception in 2007 to inform current and future programming at the country level;

3. Assess whether the results achieved are relevant for the current statistics related environment in the country and provide solid basis for UN Agencies for future programming;

4. Provide sharp and forward-looking recommendations that can guide UN agencies and other players in developing effective, realistic and evidence-based strategies.

The summative evaluation is comprehensive and covers the outcome, outputs, activities and inputs of the Project. It is expected that the results of the evaluation will be used for re-focusing the interventions and guiding future programming. In this context, the evaluation has:

(i) Extracted lessons for future interventions in the statistical sector;

7 See the Annex 2 Stakeholders consulted


23

(ii) Proposed improvement of the coordination between donor-supported interventions in meeting national requirements; (iii) Outlined main areas of focus for future UN programming.

B. Future programming in the field of statistics and evidence-based policy development, monitoring and evaluation In terms of future programming, the Evaluation Team has:

Analyzed the national priorities and needs for statistical sector institutional and functional development, formulated by relevant national counterparts (NBS, line-ministries, National Bank of Moldova, Central Election Commission);

Explored the Government’s vision (needed types of interventions) of the public administration reform in the field of statistics in order to create an efficient, functional and sustainable institutional framework for evidence-based policy-making and to ensure high quality, internationally comparable data production and dissemination;

Assessed the Government’s readiness and concrete commitments for the implementation of the national and sectoral strategic documents (in the fields of macroeconomic, employment, local and regional development, decentralization, research & development and innovation, social protection, education, environment, gender etc.) in terms of their evidence-based monitoring, evaluation and reporting, as well as interventions related to sector level statistics (developing of capacities at central and local levels, piloting new data collection tools and methodologies, policy advice support, etc.);

Estimated the interest and readiness of development partners and other stakeholders to support and engage in activities aimed at enhancing the understanding of different state agencies on reporting and monitoring of MDGs, key national strategies, and poverty and social inclusion that would contribute to improvement of national data sets;

Developed a draft Concept Note (at least outcome and output level details) on updated UN intervention related to statistical sector/field (official statistics and/or statistics produced by other CPAs) and provided concrete solutions in the context of the new RM-UN Partnership Agreement 2013-2017 and being included in the UN Partnership Framework’s Action Plan.8

Specified the roles and responsibilities of participating stakeholders (UN, government institutions, LPAs, CSOs, international partners, etc.), as well as fund management modalities (considering a joint UN intervention).

The consultants are committed to providing quality products and services, accordingly with the TOR and professional evaluation principles. As both deliverables are being developed, the consultants had at least four check points for quality control:

A discussion of the inception report and plans of action to ensure that the consultants’ understanding of what is required corresponds to Project partners’ expectations.

Presentation and discussion of preliminary findings and recommendations.

A review of a draft, or mid-point of evaluation.

An acceptance procedure for completed evaluation report and a Concept Note.

8Please see http://www.un.md/key_doc_pub/docs/UNPF%20Action%20Plan%202013-2017%20%28En%29.pdf

http://www.un.md/key_doc_pub/docs/UNPF%20Action%20Plan%202013-2017%20%28En%29.pdf


24

Adjustments were made to reflect feedback at each of these points. More informal communication contributed to the quality of the evaluation report. This process ensured that multiple opportunities are provided to resolve issues and challenges throughout the evaluation exercise.

The period covered by the evaluation was between the beginning of the Project implementation in August 2007 and the beginning of the evaluation in March 2013. The geographic coverage of the assignment is national.

The consultants followed the guidance provided in the United Nations Evaluation Group Quality Checklist for Evaluation Reports.9 The evaluation team adopted the following guiding principles so that the evaluation process and outcomes are:

Intentional so there is a clear intent to use evaluation findings.

Impartial and balanced. Impartiality is the absence of bias in due process, methodological rigor, consideration and presentation of achievements and challenges.

Independent and free from undue influence and include unbiased and transparent reporting.

Participatory as it reflected the views of as many stakeholders, Project’s beneficiaries and implementers as possible.

High quality as it will use triangulation (simultaneous use of perception, validation and documentation to analyze information).

Credible, clear and easy to understand.

Respectful of the right of institutions and individuals to provide information in confidence and ensure that sensitive data cannot be traced to its source.

Sensitive to beliefs, manners and customs of the social and cultural environments.

Evidence based and action oriented.

Gender responsive and

Future oriented and realistic in its recommendations.

3.2 Evaluation methodology This evaluation is based on the belief that evaluation should be supportive and responsive to Projects, rather than become an end in itself. The evaluation was tailored to the needs of the intended users as described in the TOR.

The consultants collected systematic information on how the Project is being implemented. The operational processes through which desired outcomes are pursued were captured and analyzed. The Project evaluation team adopted a strong participatory approach, engaging a wide and diverse range of stakeholders. As the consultants are assigned with a task of preparing a Concept Note on future programming, participation of a wide range of potential partners and beneficiaries was a necessary condition to promote broad ownership, encourage diversity of opinions, facilitate future buy-in, ensure sustainability prospects and arrive at comprehensive recommendations.

A mixed-method design was used for this complex project evaluation to ensure triangulation of data. All data gathered were verified through triangulation or ensuring the credibility of data gathered by relying on data from different sources (primary and secondary data), data of different types (qualitative, quantitative and resource information) or data from different respondents (e.g.,

9UNEG Quality Checklist for Evaluation Reports, 2010


25

beneficiaries, stakeholders, NBS management and staff, line ministries and central agencies management and staff, UN staff, civil society representatives and others).

A large set of different and complementary evidences were collected and analyzed by utilizing both qualitative and quantitative methods that included:

Data collection methods

Desk review of relevant documents was conducted. Quantitative and qualitative information was collected and analyzed. Some of the documents that were reviewed include: Moldova 2020, National Development Strategy of the Republic of Moldova for 2012-2020, Evaluation Report United Nations Development Assistance Framework 2007-2011 – Moldova, 2011; UN Joint Statistics Project – Gender Analysis, National Human Development Reports for Moldova, Project LogFrame, Progress Reports, Project and NBS statistical products, the NBS website etc. A full list of documents reviewed can be found in Annex 9.5.

Consultations with UN management and Project team were conducted to identify key informants for face-to-face semi-structured interviews, focus groups and e-mail communication and validate the evaluation methodology, questionnaires, findings and recommendations of the evaluation exercise.

A Project Theory of Change was reconstructed. It includes core Project outputs and outcomes.

Semi-structured interviews with pre-determined sets of questions were conducted. The interviews elicited information describing the processes of Project implementation, relevance, effectiveness and efficiency and impact and sustainability. Through the interviews the consultants gathered the necessary information to inform their work on the Concept Note as well. The target audiences included representatives from:

National Bureau of Statistics

Ministry of Economy

Ministry of Education

Ministry of Labour, Social Protection and Family, incl. Labour Inspection

Ministry of Health

State Chancellery

Media institutions

Civil society organizations

Congress of local authorities

UNDP

UN Women

UNFPA

UNICEF

ILO

WHO

UNCT

WB

EU


26

The selection of interviewees aimed to achieve equal representation of a wide range of Project partners and beneficiaries by key areas of Project focus: improvement of the statistical infrastructure, statistical tools, capacity building, data dissemination, data use and regional statistics. A complete list of interviewed individuals can be found in Annex 9.2.

Four focus groups of Project beneficiaries and partners were conducted. One focus group included NBS management and staff and explored the needs of the NBS as well as a range of indicators to measure progress with regard to the UN-Moldova Partnership Framework 2013-2017. The second focus group was conducted at the Ministry of Health and examined how the ministry is using statistics, which administrative data it collects and how the data are used in policy development, implementation and monitoring. The third focus group included representatives of NGOs and academic institutions that use statistics and explored general satisfaction o these groups of users with quality, timeliness, availability and accessibility of statistics. The fourth focus group included a wide representation from the UNCT to brainstorm the areas for the future Project interventions.

A focus group tool allowed gathering information from people with similar backgrounds or experiences. A small number of participants (6 to 9) were asked to answer a set of targeted questions. The questions were set in advance in a logical order to elicit participants’ opinions and experiences. The evaluators encouraged participation of all individuals invited and kept a neutral appearance through the focus group. Focus group participants could agree or disagree with each other that revealed different perspective on an issue and allowed collecting a range of opinions and ideas. The results of focus groups were recorded and analyzed to identify main areas of the future Project in the area of statistics and evidence-based policy making, delivery methods, oversight and management mechanisms, sources of funding and opportunities of partnership with other Projects.

Informal conversational interviews with questions emerging from the immediate context were conducted on ad hoc basis.

Validation of preliminary findings and recommendations with the UN Project partners. The consultants made a presentation of preliminary findings and recommendations to the UN Project partners. It validated them and provided an opportunity for UN Project partners to contribute their views and ideas to finalization of the evaluation report and concept note.

A wrap up session with the Resident Representative. The purpose of this wrap-up session was to discuss and validate core findings, concerns and recommendations and deepen evaluators’ understanding of the UN system priorities in Moldova.

The evaluation matrix has been developed by the consultants to reflect the evaluation criteria and guide the evaluation exercise. A more detailed list of questions can be found in Annex 9.1.

Evaluation Matrix

Foci Key Questions Indicators Sources of Data Methods of Data

Collection


27


Collection

UN Project Context

What are the key contextual factors (political, economic, environmental, donor and partner contexts etc.) in Moldova (at national and local levels) and in the region that should be taken into account in assessing UN joint project in Moldova?

How have these factors affected the Project’s performance?

What is the level of Government’s commitment to reforms in the field of statistics?

Level and nature of influence of external factors on the Project areas of intervention

Relevant documents

Key Government partners

Document and literature review

Interviews

Focus Groups

Relevance To what extent does the Project respond to the needs of reforms in the field of statistics and evidence-based policy making?

What was the mechanism of adjusting the Project focus and priorities? How inclusive was it? Did it work well?

Degree of congruence between UN Project interventions and stakeholder priorities

Relevant documents

Key Government partners

Document and literature review

Interviews

Focus Groups

Effectiveness To what extent did the joint UN project accomplish its intended objectives and planned results at the outcome level? At the output level?

Is there evidence that the produced outputs and outcomes contributed to the expected impacts? If so, how?

To what extent has the Project contributed to strengthening capacities of NBS and relevant line ministries/central agencies in the area of statistics and evidence-based policy making?

To what extent has the

Proportion of met planned results (quantitative and qualitative) according to the Project LogFrame/results framework

Levels of NBS management and staff, diverse data users’ satisfaction with Project interventions

Proportion of implicit institutional/ systemic changes in the field of statistics and evidence-based policy making

Concrete examples of project effectiveness

Project Reports

Project beneficiaries and partners

Other donors

Multiple reports

Interviews

Focus groups


28


Collection

Project addressed the HRBA in its programming?

What are the unintended results of the Project (positive and negative outputs, outcome and impacts as available)?

Sustainability What is the Project sustainability strategy? Which components of the Project are sustainable?

To what extent are the Project outcomes owned and sustained by NBS and other partners?

What are the key lessons that UN partners can use to inform their future programming in the area of statistics in Moldova?

Number and nature of factors that support and/or hinder sustainability of results

Partners’ assessment of Project sustainability

Levels of Government and NBS commitment to pursue reforms in the area of statistics and evidence-based policy making

Relevant documents

Key partners

Document reviews

Interviews

Focus Groups

Efficiency How efficiently have the financial resources been used?

Did the Project management ensure proper co-ordination of activities and partners to encourage synergy and avoid overlaps?

Nature and extent of cost-saving strategies in place

Degree of Project flexibility to respond to the changing environment and partners’ needs

Coordination mechanisms in place

Project staff

Project financial documentation ATLAS entries

Document reviews

Interviews

Focus Groups

Impact What is the probability that the Project will achieve its anticipated impact? What is the evidence?

Measurable impacts in such diverse areas as data collection and dissemination systems, NBS and other partners’ capacities, data users’ satisfaction levels, extent of statistics use in evidence-based policy making, etc.

Relevant documents

Key partners

Document reviews

Interviews

Focus Groups

3.3 Ethical considerations

The consultants followed the UNEG code of conduct and ethical responsibilities including guidelines on protection of privacy and conflict of interest.10 The evaluators exercised their independent judgement and provided a comprehensive and balanced presentation of strengths and weaknesses of the Project being evaluated, taking due account of the views of a diverse cross-section of stakeholders. The consultants tried to ensure that the evaluation is based on reliable data and observations.

10United Nations Evaluation Group, Ethical Guidelines for Evaluation, March 2008


29

All confidential information obtained by any means was treated in confidence. Personal, confidential and sensitive information was not discussed with, or disclosed to, unauthorized persons, knowingly or unknowingly. The interviewees and participants of focus groups were treated with objectivity and impartiality.

3.4 Human Rights and Gender Equality considerations

The consultants followed the UNEG guidance on how to integrate Human Rights (HR) and Gender Equality (GE) considerations in evaluations.11 They examined to what extent the Project benefited right-holders, including a wide range of Project beneficiaries and strengthened the capacities of duty bearers and other key players to fulfil their obligations and responsibilities. In examining the HR & GE aspects, the consultants were guided by international and regional conventions and national policies and strategies.

EVALUATION LIMITATIONS AND MITIGATION APPROACHES

As the Project was launched in 2007 and amended in 2010 and 2012, the format of annual plans and reports has evolved to reflect the changes in UNDP requirements. As a result, the Project documentation was not sufficiently consistent in its definitions of inputs, activities, indicators, outputs and outcomes that made it challenging to conduct the proper evaluation of the Project. The Project documentation for 2007-2010 did not clearly delineate inputs, outputs and outcomes and the main intended interventions and activities and expected results were not adequately defined through objectively verifiable indicators stated in measurable terms. There is some degree of ambiguity in the formulation and understanding of Project strategic objectives, which could be considered overly optimistic. Some Project baselines were not clearly identified and specific measurable indicators not developed that made problematic assessment of Project effectiveness and impact. For instance, the Project implemented numerous trainings, supported participation of NBS management and staff in international statistical trainings and events and produced numerous analytical publications, but the outcomes and effects of these activities were not systematically monitored. The Project reports often made generic statements about capacity building impact of these activities that were often not sufficiently supported with specific and tangible data and measurable facts. The Project was diverse in terms of its activities and lasted untypically long that de facto turned into the Program. The Mid Term Project Review was not conducted. Due to these factors, it was difficult to assess the Project outcomes and impacts that impoverish the results of the current and future evaluations. The evaluators acknowledge that over the last two years the entries into ATLAS became more precise and results-oriented but more work is needed to make the tools used for monitoring Project’s performance more specific, measurable, achievable, relevant and time-bound and results-focused. The Project’s risks were correctly identified and strategies addressing them were properly developed and implemented, which proves that the Project scored well on risk management and this is seen as commendable by the evaluators.

11United Nations Evaluation Group, Integrating Human Rights and Gender Equality in Evaluation – Towards UNEG Guidance, 2011


30

Through careful examination of the Project and other relevant project documentation and consultations with the UN team, the evaluators arrived to a common understanding of the assumptions of the UN team about the process through which the expected change had to occur and identified core Project inputs, activities, and outcomes and potential areas of impact.

Another limitation to this evaluation is the fact that it has been carried out in a compressed timeframe. As the Project has implemented a wide range of activities since 2007 across multiple areas and as mentioned, the mid-term evaluation was not conducted, the evaluation team had to evaluate a very complex, specific and unusually lengthy Project. One more limitation making this evaluation more challenging and different from other evaluations is that fact that statistics is a specific and complex sector covering/serving all other and different sectors. This, combined with other limitations noted in this section, has constrained somewhat the analysis and did not allow in-depth investigation of all Project activities, therefore the evaluation team decided to focus more on the outcomes and impact level.

EVALUATION FINDINGS

As mentioned, the Project is complex, with multiple and diverse activities that do not allow exploring all its aspects in this evaluation report. The following discussion presents main Project evaluation findings by key evaluation criteria. The findings are organized to highlight Project Relevance, Effectiveness, Efficiency, Impact and Sustainability as required in the ToR. The evaluation team exercised its professional judgements to identify key areas of focus and analysis.

5.1 Relevance

The relevance of the Project was analyzed in the context of the Project design, the approach, objectives and strategy, logic of intervention to address the problems and the statistical systems` development needs and Project relevance to national policies and international commitments. Thus, relevance deals with the appropriateness of the Project design to the needs of reforms in the field of statistics and evidence-based policy making in Moldova. The evaluation team examined the degree to which the overall purpose of the Project remains valid and pertinent in the current context and can provide basis for further statistics development in the country. The evaluation findings proved that the Project design is relevant vis-à-vis the overall Project goals. The overall Project design comprises a holistic, integrated and multidimensional and in-depth approach. It addresses identified statistical systems` development needs at all three different stages.


31

Figure1 shows that interventions at each component are closely interlinked and aimed at mutually reinforcing one another for greater synergy and impact on statistical system. The Project design, as such is holistic, aimed at bringing about systemic and transformational changes and capacity building by working at all three components of the national statistical system.

The analysis of the overall design shows that Project design interventions have greater emphasis at the data producing and disseminating components, and lesser explicit emphasis is on data use. In Spring 2013, when this evaluation is conducted, the Project remains highly relevant to realities of reforms of statistics and evidence-based policy making in Moldova. The Project has correctly and in a participatory manner identified and addressed fundamental limitations in the areas of statistics and evidence-based policymaking. In this context the following quotation is eloquent: “The Project is particularly interesting as support for a statistical agency is not normally the remit of UN agencies to any great extent and is non-standard area of intervention but intervention logic was high and justified. UNDP and UNFPA rightly assessed a gap and need for quality data on issues to do with population which has now expanded to data at a decentralised level across regions in Moldova.”12

Some of the core limitations of the national statistical system addressed through the Project include:

Relatively underdeveloped NBS data collection and dissemination systems insufficiently to provide timely, quality statistics to support policy development, implementation and monitoring processes. Some methodologies were inconsistent across sectors of NBS, there were multiple data sources that resulted in discrepancy in data produced and NBS staff, especially at the regional level.

12Rupinder Singh, international consultant and Igor Nedera, national consultant, Institutional Development Outcome Evaluation, UNDP and UNFPA Projects, 2010.

An in -depth approach

Data production

Data use

Data dissemination

Statistical

system

Figure1. A Holistic, Integrated and Multidimensional Approach


32

Statistical indicators and methodologies are partially aligned with the Eurostat and other European standards that limited the possibilities of effective cross-country comparisons and were not supportive of Moldova’s EU ambitions.

Statistical data were often difficult to access and obtain and use and the users did not have an effective feedback mechanism with NBS.

Many statistical indicators were not disaggregated by sex, age, social group, geographical area etc. or, if disaggregated, were not effectively disseminated in user-friendly format.

The users had limited awareness of the importance of statistical information and insufficient knowledge and skills to use data, to inform policy development in line ministries, interpret social and economic events and other purposes.

The Project design and its objectives are highly relevant vis-à-vis national policies and strategies. The Project strengthened the Government’s capacity to report on its progress in meeting a range of international commitments, including but not limited to:

Millennium Development Goals

UN Convention on the Rights of the Child

UN Convention on the Rights of Persons with Disabilities

The Convention on the Elimination of All Forms of Discrimination against Women

Beijing Platform for Action

ILO Convention 160 on labour statistics

Etc.

The evaluation team finds that the Steering Committee established by the Project is an effective mechanism to maintain its continuous relevance to NBS’ and partners’ needs. Through annual review of Project priorities, the Steering Committee identified and revised Project’s priorities and activities through extensive and frequent consultations with the NBS, line ministries and central agencies, as well as other stakeholders. The representation of key partners on the Committee ensured that the Project is country owned and driven and that its activities are well coordinated and properly sequenced.13

The political and economic realities of Moldova have changed several times through the course of project implementation and the project adjusted its interventions to support new Government’s priorities. Election of the President in March of 2012 ended the protracted political crisis that haunted Moldova since 2009 when the pro-western Alliance for European Integration (AEI) ousted the Communists’ Party from power. The new Government managed to avert economic and financial meltdown with strong support from the IFIs and wider donor community. Furthermore, it was extremely effective in rebuilding relations with key external partners of the country and forcefully committing the country to the European integration path. Since 2010 a series of key reforms have been started, such as reforms in justice, education sector or decentralization.

The European integration is a driving force behind much of the country’s reform agenda. As external environment remains favourable and Government’s efforts to advance European integration agenda have been steadily increasing, Moldova has found itself among the top performers amongst the Eastern Partnership countries. Indeed, the progress made in this process seems to have won accolades to the current Government not only from the domestic actors, but

13Minutes of the Steering Committee Meetings.


33

also from external players, most of all the European Commission. However, the recent political events could affect the prospective of Association Agreement and undermine future political association and economic integration between the EU and Moldova. However, the new Pro Europe Coalition reinforced the European perspectives of the Republic of Moldova.

In the beginning of 2012 Moldovan Parliament adopted new National Development Strategy Moldova 2020, which contains strategic vision and seven key priorities for the country’s development for the next eight years. As such the implementation of the Strategy will address such core pre-requisites for future development as: energy saving and development of renewable sources of energy, improving education outcomes, road infrastructure, and others.

Another key area is the Public Administration and Decentralization Reform, which aims at strengthening the role of local public authorities in making public expenditure more transparent and efficient, and equitably improving population’s access to basic public services, ranging from education and health to water supply and sanitation. As such, in long term the reform should help Moldova overcome the rural-urban divide that plagued country's progress in recent years. For instance, the disaggregated data outlines the persistence of rural-urban divide.

As a result of these broad political changes and increased interest of policymakers to substantiate their decision with some evidence, NBS has become more increasingly called upon to provide more comprehensive and sophisticated statistics and the Project strengthened the NBS’ capacity to do so. To support the Government’s priorities, the Project focused on strengthening the users’ skills in data use in policymaking and monitoring, approximation of many indicators and data collections processes with Eurostat and broad EU requirements and improving the quality and availability of data necessary to support the country reforms.

The high Project relevance was acknowledged by the interviewees and participants of four focus groups. Its relevance is demonstrated by the use of leading indicators by line ministries, State Chancellery, NGOs, academics, international development partners and the media that indicates their confidence about data quality. The evaluation team found, however, that statistics are mostly used for reporting on Moldova’s progress with regard to meeting its international commitments and broad Government programs, while the use of statistics to inform policy, programming decisions and monitoring the current policies and estimating their impact on population still remain limited. The Project relevance was also confirmed by other international development partners, such as the World Bank. The evaluation also showed that the Project was flexible and responsive to partners’ needs. For instance, in the original project’s plans, the ecological statistics were not covered but during the project implementation the need for improved ecological statistics was realized and the Project responded to it. In this purpose, the Project hired two experts (highly appreciated by NBS) on ecological statistics that helped respective NBS staff to adopt the indicators on ecological statistics and improve the methodologies used and pertinent information flows. A joint NBS and the Ministry of Ecology working group was established that reviewed the environmental statistics and produced a joint country report on ecological issues.

The State Chancellery of the Government is strongly promoting the ex-ante policy analysis among line ministries that requires using of effective, accurate and reliable statistics for Cabinet submissions and other policy documents. In the evaluators` opinion, the Project correctly identified a need to focus more


34

efforts on building line ministries’ capacities as main statistics’ users and is planning a series of capacity building initiatives in this area, if a new Project in the area of statistics is implemented. Finally, the Project was designed according to international norms and agreements on Human Rights (HR) and Gender Equality (GE) and in line with national strategies to advance HR & GE.Human rights and gender equality frameworks were adopted in the Project design, in particular in supporting NBS in development of disaggregated data by gender, social groups and localities, producing gender related statistical and analytical products (“Women and Men” publication, “Violence against Women” Report, Social Exclusion, education statistics publication, Report on situation of Roma population, Use of Time by women and men, etc.),. The increased availability of disaggregated data activities strengthened the state and society capacities to respect and protect human rights, as well as fulfill human rights by taking steps progressively to realize them.

5.2 Effectiveness

The following part of the report presents the findings of the final evaluation related to the effectiveness of the Project. Effectiveness focuses on results, not processes and the evaluation team examined if the Project has produced its planned outputs and outcomes. Multiple sources of evidence were used to examine if the Project is achieving its intended objectives.

The Project strengthened routine data collection, storage, compilation, dissemination and statistical analysis, and improved gender, social, population, labour and other statistics within the NBS. It implemented a range of activities to improve statistics’ use by policy-makers and other key users, as well as made data access and dissemination consistent with the international best practices.

As mentioned in the relevance part of the report, the Project is logical and targets three key components: data production, dissemination and use. Its specific results were updated on yearly basis to reflect NBS and other partners’ needs. In evaluators’ view, the Project’s comprehensiveness represents one of the key success factors that contributed to Project’s achievements because statistics could not be improved by addressing only data production and dissemination but should equally emphasize the ultimate users of statistics. Again, the Project design was broad enough to facilitate a variety of activities and provided the management with a significant level of flexibility in responding to partners’ needs. As specific results presented below demonstrate, the Project contributed to: a) improving the statistical infrastructure; b) improving statistical tools and methodology; c) data dissemination; and d) data use.

The findings of the final evaluation prove that the Joint UN Project scored well under the effectiveness aspect because the planned results have been successfully achieved and in some areas even exceeded. The evaluation team tried to separate Project’s inputs, outputs and outcomes to present and demonstrate the role of Project interventions. Thus, the Project’s core outputs and outcomes since 2007 to 2013 include:

NBS’ capacities have been strengthened in a number of areas:


35

A number of recommendations were provided to support NBS functional optimization. The analysis was conducted and advice provided on optimization of organizational and personnel structures of the NBS in the broader context of public administration reform.14

The database on demographic statistics was established. A set of modern equipment and the necessary software were procured that helped to transition from manual calculations of demographic statistics and facilitated forecasting in the Demographic Statistics Section. The analysis of primary data on vital events and assessment of their quality was undertaken by a consultant hired by the Project and recommendations were developed for NBS consideration.15

Data collection processes were improved. NBS was provided with “Unitary rules on the design and editing of statistical methodologies” developed by the Project. The Project provided training to NBS network of 201 interviewers and observers through 12 thematic trainings that improved the processes of implementation of households’ surveys which are some of the most important data collection processes.

A number of tailored recommendations were provided to improve E-development statistics. NBS was supported in developing a comprehensive list of indicators for e-development that was aligned with the latest international standards and best practices in e-development. The Project provided also legislative and operational recommendations on optimization of distribution of functions and responsibilities among the official authorities (ministries, NBS, ICT national agencies) who participate in the collection, production and dissemination of e-development indicators.

Environmental statistics were improved. The Project hired two consultants who reviewed the statistical infrastructure and toolkit in 3 fields of environment, namely water resources, air pollution and waste management. As a result, the positive effects of the Project interventions are evident because the system of indicators and toolkit in 3 fields of environment has been revised and separation data collection responsibilities in separate areas of environment statistics among the concerned parties were facilitated.

Statistics to support effective decentralization were inventoried/analysed. With the support of the Joint Project a comprehensive report on decentralization and vulnerability indicators was drafted by a team of international and national consultants.16 It captures the available and needed statistics to develop, implement and monitor decentralisation. An extensive list of indicators to monitor the decentralization and its impact on population, including vulnerable groups was developed and shared with NBS and State Chancellery.17 The usefulness and policy relevance of the list was highly evaluated by the partners and other interviewed stakeholders. A highly appreciated workshop on monitoring decentralization was conducted with participation of the international expert and representatives of central and local public authorities, National Bureau of Statistics, Local Public Authorities Congress, development partners, civil society, academia was conducted.18 But the produced report and set of

14 Terms of References International consultant to support the National Bureau of Statistics with optimization of its organizational structure, 2010 and reports submitted by the consultant. 15 Terms of References, National consultant to undertake the analysis of primary data on vital events and assessment of their quality, 2012; Tatiana Medvediuc, Efectuarea Analizei Datelor Primare Privind Evenimentele Vitale şi Evaluarea Calităţii Acestora, Nota Analitică, 2012. 16Decentralization and Vulnerability Indicators: Assessment report on data collection, capabilities and potential, Republic of Moldova, 2012 17List of indicators of monitoring the decentralization and its impact on population, including vulnerable groups, 2012; 18 Workshop on “Monitoring the decentralization reform and its impact on population, in particular the vulnerable groups”, 2012


36

indicators represent a practical toolkit which is meant to support the CPA and LPAs, under the general coordination of State Chancellery, to support with evidences the policy-making and monitoring of decentralisation strategy

The labour statistics were significantly enhanced as a result of the impressive initiatives supported by the Project. The Project and UNDP played an important role facilitating and supporting the collaboration of NBS with ILO Geneva. Particularly, an ILO expert was engaged and helped to improve the statistical infrastructure in the fields of labour market and living standards. The reporting time and efforts with regard to country’s progress regarding national and international documents and treaties has been improved as well. Another ILO expert provided practical support necessary for the preparation of ratification of ILO Labour Statistics Convention no. 160.19 Other ILO experts supported the development of the methodology of the survey on Child Labour. The Project also supported human resources development initiatives, for instance 201 interviewers and observers were trained and acquired knowledge and developed practical skills on how to conduct specific surveys in the field; NBS staff was trained to coordinate a Child Labour survey; the information notes on child labour issues and flyers on child labour issues were developed and disseminated. The toolkit of statistical survey on accidents at work with regard to its compliance with applicable international standards and European norms was reviewed by the National Consultant hired by the Project and recommendations were shared with the NBS, Labour Inspection and Ministry of Labour.20 The Project supported development of the methodology and tools for the Labour Migration Survey. The National Consultant was hired to revise the forms of primary accounting on labour time and wages for statistical purposes.21

The NBS statistical toolkit for the survey on enterprises was harmonized with the requirements of the National Accounting Standards. Again, the consultant hired by the Project provided valuable recommendations on how to harmonise the NBS statistical indicators with the requirements of the National Accounting Standards.22 As the immediate effect of the intervention, it helped to incorporate the questionnaires into the standard accounting software applications used in Moldova, increasing, in such a way, the effectiveness of the tool.

A number of recommendations in the area of Research and Developments statistics were provided, mainly because the international consultant, provided by the Project, evaluated the general situation in the R& D statistics and proposed improvements on information flow and the system of statistical reports (data) for the production of R&D statistics.23

Support in preparations for the Population and Housing Census 2014 was provided through advocacy and methodological support (e.g., improving content of 2 questionnaires).

19 Igor Chernyshev, ILO Department of Statistics, Review of the national labour statistics in relation to the application of the ILO Labour Statistics Convention 160 (1985), 2010 20 Terms of References, National Consultant to Revise the Toolkit of Statistical Survey on Accidents at Work in Compliance with Applicable International Standards and European norms, 2012; VioricaGhimpu, Raport detotalizare a lucrarii cu privire la revizuireainstrumentelorstatisticenaționale de înregistrareșiraportare a accidentelor de muncăînconformitate cu standardeleinternaționaleșinormeleeuropeneaplicabile, 2012 21 Terms of References, National consultant to revise the forms of primary accounting on labour time and wages for statistical purposes, 2012 22 Terms of References, National consultant in accounting to expertise and test the statistical toolkit (questionnaires) to be applied within the integrated statistical survey on enterprises, 2012 23Marek Kwiek, Strengthening Research and Development (R&D) Statistics in the Republic of Moldova and Final report on undertaken assignment, 2009


37

NBS was supported in developing its website. The Project’s consultants reviewed the international standards for dissemination and exchange of statistical information and developed technical requirements and internal regulation on the maintenance of NBS website.24 As the result, the website design and user-friendliness were dramatically improved that was appreciated by multiple users interviewed through the course of evaluation.

A number of data users’ satisfaction surveys with available statistical data produced by NBS were conducted. The survey was institutionalized as a mechanism for obtaining regular public feedback on the development of the national statistical system and stimulating dialogue with data users.

A wide range of diverse NBS publications were supported that include in particular statistical compilation “Labor force in the Republic of Moldova. Employment and Unemployment 2008”; statistical compilation “Criminality in the Republic of Moldova 2010”; Statistical compilation “Aspects of living standards of population in Moldova 2008”; “Population and demographic processes 2008”, Statistical data and Analysis on “Conditions of Enterprises Start-up and Development: Analysis through gender lens 2009”; Practical Guide for the interviewers of HBS and LFS; statistical compilation "Education in the Republic of Moldova (2010, 2012)"; “Approaches to social exclusion in the Republic of Moldova. Methodological and analytical aspects” 2010,“Children of Moldova” (2008 and 2012), annual thematic calendars with statistical data (2010 on MDGs, 2011 on social exclusion, 2012 on regional statistics, 2013 on quality of work) and many others.

Statistical methodologies in many areas were aligned with the EU standards that facilitates cross-country comparisons and supports Moldova’s EU integration processes.

National definitions and calculation methods for 5 MDGs indicators were updated and harmonized with the international standards and disseminated due to the Project support. A number of reports were produced by international statistician experts on measuring and monitoring MDGs.25 Ministries’ policy monitoring units were targeted in training.

The Project provided expertise and support to ensure compliance of statistical methodology and toolkit for Time Use Survey, Integrated Statistical Survey on Enterprises, Research & Development statistics with the Eurostat and other international standards and good practices.

The Project hired an international consultant to perform an in-depth assessment of distribution of incomes and living conditions statistics and their adherence to the EU Statistical Requirements Compendium. The income and living condition area is subdivided into four further domains: Household Budget Survey (HBS), Income, Social Inclusion and Living Conditions. The report and recommendations were shared with the NBS.26

24Activity Report on consultancy provided for the National Bureau of Statistics aiming to develop e – Statistics Web Portal Concept (final version) during the period of 25.04.2008 – 20.05.2008; Provision of consultancy for the improvement of the data dissemination tools and practices for the National Bureau of Statistics, Activity Report 25Government of Republic of Moldova and UN Moldova, Improved measurement and monitoring of MDGs in Moldova: targets, indicators, definitions, data sources, progress analysis: Results of technical support missions by specialized statistician experts (MDG 7: Ensure environmental sustainability by WHO expert, Rifat Hossain; MDG 3: Promote gender equality and empower women by ILO Expert, Adriana Mata-Greenwood; MDG 2: Achieve universal access to general compulsory education by UNESCO expert, Saïd Ould Voffal; MDG 4-5: Reduce child mortality and improve maternal health by Fern Greenwell, ex WHO expert) 26Markku Lindqvist, Finland, In-depth assessment of the income statistical field of the republic of Moldova and its adherence to the EU Statistical Requirements Compendium, 2012.


38

The Project hired a consultant to conduct an in-depth evaluation of population statistics in the country in order to provide a detailed assessment of the system’s strengths and weaknesses in this area. Recommendations on such areas as vital statistics, migration and asylum, population census and population projections and gender were provided to the NBS.27

Concepts, methodologies and questionnaires for a number of statistical surveys (e.g., labour costs,28 labour time & wages, annual structural statistics, work accidents29) were revised to ensure compliance with the national legal and regulatory framework and needs of key stakeholders, as well as the EU/international standards and practices.

NBS management and staff skills and capacities were strengthened through targeted training and international exposure. Some targeted capacity building interventions implemented by the Project include:

A continuous training program for statistical personnel in regions has been improved due to new curriculum content and teaching materials for module on Interviewing Skills & Abilities developed by the Project for the regional offices of NBS.30

A number of training of the interviewers has been conducted in the framework of the sample surveys on households.

The Project supported also Training of Trainers of NBS regional staff conducted by experts from the Office for National Statistics (ONS) of Great Britain. As a result, a selected team of training beneficiaries can conduct the trainings themselves based on professional needs of the specialists involved in the statistical survey. The training was very highly evaluated by all participants.31

Publication of the internal Informative Bulletin regarding the activity of the LFS, HBS and agricultural statistics networks of interviewers has been supported and this contributed to increasing the visibility of the NBS` activity and dissemination of data.

NBS management and staff benefitted from first hand exposure to experiences of the EU partners in implementing Eurostat and international standards. The evaluation revealed that they learned successful practices in statistics and shared their own achievements with colleagues from other countries. Some examples of participation in international forums and training opportunities supported by the Project include:

Exchange visit to the Central Statistical Bureau of Latvia on implementation of integrated information system.

Participation in the 12th Inter-Agency and Expert Group Meeting on MDG Indicators, Paris.

Re-establishment of the bilateral collaboration with NISEE (National Institute of Economic and Statistical Studies), Paris.

Participation in the Regional Workshop on international migration statistics in Geneva.

Study visit on demographic statistics to the Statistics Department of Lithuania.

27Jean-Michel Durr, international expert, In-depth assessment of Population Statistics of the Republic of Moldova and its adherence to the EU Statistical Requirements Compendium 28 Terms of References, National consultant to develop the methodology and tools for the Annual Statistical Survey on Earnings and Labour Cost in compliance with the international standards, 2012 29Viorica Ghimpu, consultant UNDP Moldova, Report on the revision of national statistical tools on recording and reporting of occupational accidents in compliance with applicable international standards and European norms 30See, for instance, Ludmila Malcoci, Guide for interviewers (in questions and answers), 2008; and Ludmila Malcoci, Curriculum for the training of interviewers, 2008 31 Final report for ‘Train the Trainers’ 23rd – 27th November 2009


39

Regional Training of Trainers on gender statistics in Kazakhstan.

Participation in the Global Forum on Gender Statistics in Rome, Italy.

Regional Meeting on Gender Statistics Sensitization Programs, Rome, Italy.

Attendance of the joint UNECE/Eurostat Work Session on Migration Statistics in Geneva.

Attendance of the meeting on the Management of Statistical Information Systems and Work Session on Statistical Metadata in Luxembourg organized by EUROSTAT, OECD, UNECE.

Participation in the Conferences of European Statisticians.

Participation in a 3-weeks intensive course on ‘Demographic Analysis and Software’.

Participation in a Joint UNECE/ILO/EUROSTAT Meeting on the Measurement of the Quality of Employment in Geneva.

Attendance of the UNECE Regional Expert meeting on measuring violence against women during in Geneva.

Participation in the Expert Meeting on Collecting and Using Data to Demand Effectiveness of Policies against Violence against Women in Sarajevo.

Mission to the National Institute of Statistics of Romania.

Participation at the UNECE sessions on gender statistics, including violence against women.

Participation at the Regional Seminar on international migration statistics, Bishkek, Kirgiz Republic.

Participation at UNICEF Network meetings of National Statistical Offices correspondents for Trans MoNEE 32

All above information highlights the efforts of the UN Joint Project to support the process of the strengthening of the national statistical system.

Metadata production and dissemination was supported. Capacities of NBS in production of metadata have been strengthened. The Project hired two consultants who prepared a standardized set of metadata. As a result, metadata for 8 statistical fields and 28 core statistical indicators that derived from national MDGs were produced and made publicly available.33 The evaluation team reviewed the metadata and sectoral data34 available on the NBS website and confirm that the metadata supported by the Project are of high quality, user-friendly and provide rich information. The interviewees from line ministries and central agencies were highly satisfied with the quality and accessibility of the metadata which demonstrates the quality and accessibility of these products.

Social statistics production, dissemination and use have been significantly improved. Social statistics indicators have been significantly expanded.

An inter-ministerial working group consisting of representatives from concerned ministries and NBS was established to coordinate the process of development and analysis of social exclusion indicators. National experts were exposed to experiences of Poland35, Croatia, and Romania and benefitted from the expertise of UNDP BRC.

32 TransMoNEE – Transformative Monitoring for Enhanced Equity, UNICEF supported initiative aimed on improvement of data collection on the situation and wellbeing of children, young people and women in countries of Central Eastern Europe and Commonwealth of independent States (CEECIS). 33 See, http://www.statistica.md/pageview.php?l=ro&idc=430& and http://www.statistica.md/pageview.php?l=ro&idc=433& 34The metadata were produced for 8 MDGs and a wide range of sectors such as agriculture, national accounts, labour force, industry, education and science and others. 35 Measuring social inclusion at sub-national level: Lessons learned from Poland, Based on study tour for Moldovan civil servants, 16-20 June 2008




40

The data sources, including administrative data in relevant areas (e.g. social protection, vulnerable groups, employment, disability,36 civil registration, decentralization) were reviewed and analyzed and recommendations on improving data production were made and, as mentioned one of the interviewed stakeholders “those recommendations proved to be extremely helpful for us because they really helped us to improve our data production”.

A number of statistical publications and thematic materials (leaflets, synthesis, analytical papers, etc.) on social topics were supported and produced in user-friendly and easy-to-use format and widely disseminated. The Project published the Approaches to Social Exclusion in the Republic of Moldova that provides the tools, including disaggregated statistics, to monitor social inclusion and identify policy and programs priorities. The report development strengthened NBS capacities in the area of social statistics (e.g., social exclusion). The Report was successfully launched and key line ministries, State Chancellery, representatives of academia, local think-tanks, media, and international organizations attended the launch event. About 300 copies of the report were disseminated, which represents a good opportunity for raising awareness and visibility towards Project related issues. The evaluation team acknowledges high quality and user friendliness of all of the Project publications, including calendars (e.g., Calendar with the information on social exclusion that presents key statistical information on social exclusion,37 Calendar with statistical indicators on Moldova’s progress with regard to MDGs38) and several Project and statistics issues related brochures.

It is also important to mention that the final evaluation findings prove that the statistical infrastructure and processes of data production and dissemination in the field of justice and criminality have been improved. Relying on the advice provide by the consultant hired by the Project and validated by key ministries, a Memorandum of collaboration was signed by the NBS and Ministry of Internal Affairs, General Prosecutor’s Office and Ministry of Justice comprising conditions and terms of statistical data presentation.

Statistical data disaggregated at raion and community level have been collected by the Ministry of Economy and used for calculation of the Small Area Deprivation Indexes calculated by the Ministry of Economy and disseminated through DevInfo (2007-2010) and, as a consequence it improved the poverty monitoring framework.

Evaluation also revealed that a mechanism of routine data collection on children in institution and alternative services was established. The system of data collection on children in institution and alternative services has been reviewed (in cooperation with the MLSPF), statistical forms approved and regular data collection will be carried out by the MLSPF starting from March 2013

Gender was mainstreamed into multiple areas of statistics, capacities of NBS and data users were strengthened in gender statistics and high quality analytical products were produced.39

A set of harmonized gender sensitive indicators was developed through multi-stakeholder consultations that involved more than 100 partners. The matrix of gender indicators was included into the National Strategy on Gender Equality of Moldova and was extensively used to monitor country’s progress with regard to a number of national and international commitments.

36Report on revising statistical toolkit in the field of disability statistics in Republic of Moldova 37 Social Exclusion in Moldova through official statistics, 2011 38Millennium Development Goals and derived statistical indicators, 2010 39More activities and results can be found in the Project Annual Plans.


41

For instance, NBS due to the Project support has developed and harmonized a set of gender-sensitive development indicators (MDGs, the Convention on the Elimination of All Forms of Discrimination against Women (CEDAW), and Beijing Platform for Action (BPfA).40The process of harmonization of gender-sensitive indicators was identified by experts as a successful practice and was replicated in several countries in the CIS and it can be perceived as a best practice.

An important and long lasting achievement in the opinion of the evaluators is the fact that the capacities of the NBS to conduct gender-specific surveys have been strengthened. The survey ‘Violence Against Women in Family’ was designed through consultations with various partners. The international consultant provided extensive support and shared experiences of other countries in conducting similar exercises. The software application was developed41, survey was conducted and complimented by a series of focus-groups and in-depth interviews.The data on violence against women were collected, disaggregated, analyzed and used for advocacy and policy making. Representing the first ever official data collected by NBS on this topic, the results of statistical survey have been widely disseminated through a separate analytical publication and thematic leaflet used during the national campaigns dedicated to fight against domestic violence. Also, through this survey NBS was one of the few countries that piloted the pertinent statistical toolkit drafted by UNECE and has been brought as example of good practice to other national statistical offices.

Gender was mainstreamed into numerous surveys undertaken by the NBS and represents an eloquent example of the successfulness of the Project. Time Use Survey, first ever carried out in Moldova according to EUROSTAT methodology, provide the necessary information for the analysis of the ways in which the time is used by the population, women and men, for different purposes – personal care, economic activities (principal and secondary), education and training, mutual help, cultural activities, sports, social relations and associative activities, etc. The gender perspective was incorporated into statistical surveys related to population/households (sex disaggregated statistics resulted from the Household Budget Survey and Labour Force Survey) or on enterprises (sex-disaggregated data on entrepreneurship).

NBS’ capacity in data collection and processing of Enterprises Statistics (sex-disaggregated data on entrepreneurship) through gender lens has improved. The software and the related documentation were developed. The Survey on “Conditions of initiation and development of enterprises in gender aspect” has been conducted.

“Women and Men in the Republic of Moldova” analytical report was developed and published (2008 and 2012. in Romanian and English).Pocket books “Women and Men in the Republic of Moldova”, “Gender Statistics User Guide”, “Population: Definitions and Indicators Guide”, “Harmonized set of development indicators in the context of the MDGs” were published. The evaluation team found the above mentioned publications easy to understand, comprehensive and analytically sound.

Seven practical workshops for gender focal points/councils in line-ministries aimed to adequately use in practice the sector sub-sets of indicators were conducted and 34 specialists

40 UNIFEM, Promoting Gender Equality in National Development Policies and Programmes in Moldova; Final Report to the Swedish International Development Cooperation Agency (SIDA), May 2007-April 2010 41 Ion Amarfii, Activity Report, Development of Software Application for the survey “Violence against Women in the Family in Moldova”, 2010


42

from Local Public Administration were trained to use gender statistics in policies development42 and this represents a good example of the capacity building efforts undertaken by the Project.

Another six workshops on monitoring of 2011 Action Plan of National Program of Gender Equality Assurance implementation were conducted. They were customized to labour, health, education, social protection and other sectors. A national consultant was placed in the Ministry of Labour, Social Protection and Family to strengthen its and Local Public Authorities’ capacities to use gender statistics for the monitoring and evaluation purposes for effective implementation of the National Program on Ensuring of Gender Equality43, as a result this initiative helped to strengthen not only the monitoring skills of the targeted group but also to consolidate the capacities of the line ministries and facilitated expertise transfer.

Capacities of statistics’ users have been strengthened.

The Project conducted training for participants from line ministries on practical skills in using and interpretation of social exclusion indicators. About 50 data users were trained through 5 tailored trainings for different categories of users on how to use statistics in decision-making (e.g., media, policy monitoring units from the various ministries, independent think-tanks, etc.).

About 180 public servants (ministerial policy monitoring units, social assistants from LPAs) were provided with hands-on training on how to use statistics to design and monitor policies and progress in implementation of Government’s programs (8 workshops, 1 round table, 1 on job-training).

A number of public awareness events that demonstrated the importance of statistics to decision-making were conducted. Some of these events featured results and findings of the conducted surveys or analytical papers. The events targeted representatives of NBS, respective line ministries, representatives of local NGOs, think-tanks and academia, international organizations. In 2011, for instance, the Project conducted a workshop for journalists on documentation, use and interpretation of official statistical data.44

State University of Moldova, Department of journalism and mass media, was supported in developing courses (including curricula, manuals, guides for lectors) with special focus on statistical notions and process, documentation, interpretation and use of statistical data in the journalistic/communications activity/profession.45 It is expected that the courses will be launched in September 2013.

The Policy Research Component of the Project supported research Projects. Out of 92 research topics submitted, four research Projects of local NGOs were supported. The following research publications resulted from this research: “Analysis of the Health Status through Statistical Indicators”; “Situational Analysis of Labour Force in Rural Areas through Gender Lens”; “Analysis on Direct Foreign Investments” and “Impact of Prices on the Wellbeing/

42The Use of Gender Statistics is Essential for their Daily Job at the Local Level. In The UN in Moldova Magazine, April-June 2012 43Report of activities of national consultant to support the Ministry of Labour, Social Protection and Family in coordination with national stakeholders on the effective implementation of the National Program on Ensuring of Gender Equality 44 Journalists trained to inform people through statistics’ lens, http://www.undp.md/presscentre/2011/StatisticsTraining_21May/index.shtml 45 Call for Expression of Interest on development and institutionalization of statistical literacy university course for journalists and communication specialists, 2012


43

Wealth of the Population”. The studies demonstrated the value of quality statistics and contributed to development of evidence-based decision-making culture in respected ministries. The evaluation showed that the analytical products proved to be of high quality and demonstrated in practices the importance of statistics for policy development, implementation and monitoring.

The consultants made the following general observations regarding the Project effectiveness:

Socio-economic and political conditions in the country were unstable during the Project implementation that negatively affected its effectiveness in the field of building users’ capacities. The global economic and financial crisis and political instability undermined political commitment to reforms of the national statistical system and reduced the budget allocations to NBS and other key partners. In spite of these challenges, the Project objectives were exceeded in multiple areas.

The mere number of activities and results achieved by the Project demonstrates that it gained respect of key partners that entrusted the Project with significant responsibilities. It addressed a complex and wide range of thematic areas such as gender, demographic statistics, labour market, social exclusion, environment, e-development and many others. The Project’s flexibility and responsiveness to the Government and NBS’ needs (e.g., support of public administration reform, decentralization, European integration, extensive capacity building of NBS staff) served as important success factors and positively contributed to its effectiveness as the Project could re-orient its activities to focus on high priority areas.

The evaluation team found that the Project adopted an appropriate approach of providing support to all elements of the national statistical system, “thinking out of the box” and going beyond NBS. Improvements were needed in all parts of the system, including line ministries and other producers and users of statistics. As a result, the Project components well interact and complement each other, especially in the fields of gender and social exclusion statistics.

The project’s effectiveness was highly appreciated and evaluated by NBS management and staff and many other partners and statistics users. The evaluation team heard predominantly very positive feedback about the Project. The interviewees provided numerous examples illustrating that departments of the NBS benefitted and improved their capacities in terms of knowledge, skills, tools and methodologies in data collection, data analysis and data dissemination.

Some Project activities increased line ministries’ capacities to use statistics in policy development, implementation and monitoring, but they frequently lack the necessary knowledge and skills to use statistics in the policy cycle, including such areas as conducting research, developing policy options, conducting ex-ante policy assessments as well as estimating costs of new policy initiatives.Although many ministries have Division of Policy Analysis, Monitoring and Evaluation (DAMEP), their levels of competence in collecting administrative data and using all the available data in policy development, implementation and monitoring are very different (e.g., the Ministry of Education started to use more actively the data produced by NBS, for instance on child employment).There is also limited demand from the Parliament for evidence and statistics in reviewing draft legislation submitted for consideration. The expertise of local governments in using


44

statistics and other data in their decision making is weak and insufficient to support decentralization pursued by the Government. It applies in particular to using social statistics at the local level to develop a better understanding of issues related to social exclusion and adopt effective policy and programmatic responses to address them. It seems that the goals of building statistical culture are not adequately prioritized by the Government and not yet embedded in the broader context of public administration reforms aimed at building a strong, effective, and efficient public administration system.

One of the main winning strategy and vehicles of supporting NBS and other partners used by the Project was hiring short-term consultants to meet specific time bound deliverables (e.g., develop training manuals, review statistical methodologies and processes, produce analytical reports on the basis of statistical data). A proper mix of national and international consultants was maintained. The evaluation team found that the approach of hiring experts from new EU member states was effective in terms of transforming the knowledge and expertise to the counterpart staff. The contributions of many international and national consultants have been highly commended and the overall services of most consultants hired were of excellent quality. The NBS staff was highly satisfied with their performances in such diverse areas as labour, demography and social statistics. The solutions, methodologies and recommendations made by consultants hired by the Project were pragmatic and in line with Eurostat and international standards that have strongly contributed to the sustainability of the Project outcomes.

As can be seen from above mentioned interventions, the Project provided extensive multiple and impressive capacity building interventions for diverse groups of beneficiaries. Those interviewees who benefitted from these opportunities highly evaluated relevance, usefulness and applicability of supports received. For instance, participation in international conferences and training on gender helped to improve the indicators used in Moldova and training on sampling helped to improve sampling methodologies. The Project does not have yet, however a comprehensive system of monitoring the actual relevance, usefulness and impact of these interventions. The Project implemented numerous training interventions, but participants’ satisfaction surveys and follow up assessments were not systematically conducted. One example where a very simple survey of participants was conducted is the Training of Trainers of NBS regional staff conducted by experts from the Office for National

What do the data users said? The quality of statistics improved but still there are many cases of discrepancy between the NBS` statistics and the administrative data of line ministries. ----------------- I am very satisfied with the seminars provided by the Project because I have learned a lot about such programs as SPSS. ------------------ NBS improved its website, now it provides more information and is better accessible. ------------------ Of course we need training courses in using the data. ...................... No, I do not need any training courses in using the data; I know how to use it. I just need timely, accurate and disaggregated data. ------------------ We are journalists and we do not know how to use all the data effectively. ------------------ We need access to the primary data.


45

Statistics (ONS) of Great Britain. A list of questions was too simple to inform further training, for instance “The course met its objectives”, “I learned new skills”, “I would recommend this learning activity to a colleague”. In general, the training was very highly evaluated by all participants.46

Partially due to the Project interventions, users’ satisfaction with NBS’ products have improved. According to two user satisfaction surveys, NBS’ independence and expertise is well acknowledged by the public. Among all government services the NBS enjoys a high level of trust.

The Project managed to improve signficantly NBS’ social statistics field, including introduction of new and multi-dimensionally disaggregated indicators aligned with the European and international standards and producing solid analysis that affected the actual policy processes. Transparent, reliable, timely, and comparable estimates of poverty, inequality and other social inclusion indicators are essential for a wide range of policy and program decisions. “UN has played a key role in enabling disaggregation of national data by sex along gender and other categories, and ensuring that the data is widely available to feed into policies and programs for better outcomes. The Joint Project “Strengthening National Statistical System of RM” was instrumental in this field.47The evaluators believe that in both social inclusion and gender statistics the Project’s approach was the most effective and innovative as it included all stages of statistics from indicators definition, data production, quality control, dissemination to solid analysis and policy development and monitoring.

The data and metadata bases and development of electronic data dissemination tools accessible on-line to the general public ensured transparency and easy access to data and built sustained demand for quality, timely and accessible statistics. A next logical step for NBS would be to build internal capacity to produce more analytical interpretation of the data in the forms of reports, analytical papers, and briefings to contribute to evidence-based policy making. The latest statistics users satisfaction survey found that users are concerned about significant discrepancies between the data of the NBS and line ministries and other data providers such as National Bank, for instance.

Other areas where the project effectiveness can be improved are: better alignment of data produced by NBS and line ministries, improved timeliness of data dissemination, provision of more disaggregated data, and better institutionalized collaboration of NBS and line ministries.

In conclusion, effectiveness of the Joint UN Project at the level of components related to strengthening the national statistical mechanisms assessed as high, in the first place due to the wide, diverse and results oriented capacity building interventions and improving the evidence based policy making, of different ministries for different types of reporting on national and international commitments of the Moldovan Government. Project-supported activities have positively impacted the capacities of the NBS to develop new statistical tools, more relevant indicators, better methodologies, while the multi-stakeholder approach had a positive effect on the collaboration between the NBS and line ministries and other users.

46 Final report for ‘Train the Trainers’ 23rd – 27th November 09 47Evaluation Report United Nations Development Assistance Framework 2007-2011 – Moldova, 2011, http://www.un.md/un_res_coord_sys/docs/UNDAF%20Evaluation%20Report%20-%20Moldova.pdf.

http://www.un.md/un_res_coord_sys/docs/UNDAF%20Evaluation%20Report%20-%20Moldova.pdf


46

Interventions developed and implemented in the scope of the Project have proved to be necessary to sensitize the decision-makers, international development organizations and the general public alike, so as to make sure that Project efforts and their effects reach a “fertile ground”. Given the low levels of understanding and importance given to statistical issues among the general population, the impressive results achieved by the Project, should be further promoted and new development initiatives designed.

5.3 Efficiency

Efficiency is associated with the resources used to produce desired outputs. The evaluation team examined the extent to which the Project is producing its planned outputs in relation to expenditure of resources. In addition to exploring efficiency in resource use, the evaluators examined what specific steps were undertaken to optimize the Project efficiency.

Efficiency measures how economically resources or inputs (such as funds, expertise and time) are converted to results. The Project exceeded its objectives within established timeframes and costs and has demonstrated good efficiency. The Project strived to meet its expected outputs by looking for cost-efficiencies in Project operations and utilizing prudent business practices. The Project contains elements of soft assistance (e.g., reviews of statistical methodologies, trainings) and these soft elements are embedded into various Project components that makes the application of conventional efficiency indicators to these areas not feasible.

The consultants were unable to compare the costs of similar projects elsewhere as UN agencies rarely implement the projects supporting the statistical systems in transitional countries. The Project is unique (e.g., application of UN as one model, comprehensive nature and long duration) that makes its replication elsewhere problematic. Hence, the evaluation team did not analyze if it is worth doing such as a project from a cost perspective in other relevant countries of the region.

The evaluation team does not believe that the costs of Project deliverables can be lowered while still achieving Project objectives. The alternative delivery methods such as hiring permanent statistics experts placed in the NBS would be more costly and not responsive to evolving needs of the NBS and other partners. The evaluation team found both operational and financial documentation easy to understand and it is well kept, given the extent of challenges associated with managing a Project with pooled funding.48

The following steps were undertaken by the Project management to ensure quality and cost-effectiveness of the process of transforming inputs into outputs and outcomes:

UN Project office was located in NBS that did not require renting of Project premises and resulted in substantial savings over 7 years.

Partners applying for grants to produce statistics-based analytical products went through a rigorous process of selection by a representative committee and were required to make their own contribution to their project costs to qualify. The grant program was transparent with an open call for proposal and clearly defined eligibility criteria and the application process.

48See, for instance, Project Annual Work Plans and Budgets.


47

Open, transparent and value-for-money procurement processes were utilized for all procurement and human resources processes.

Internal UNDP, ILO, UNICEF and other international organizations’ experts and consultants as staff members with significant expertise in statistics and other relevant fields were often mobilised to provide technical assistance with no remuneration being paid.

The experts with significant knowledge of comparable jurisdictions (e.g., Croatia, Poland) and relevant language skills (Romania) were mobilised that resulted in cost savings and more effective advice based on the recent experiences of EU accession processes.

All consultants, national and international, were involved on short-term, as needed basis instead of long-term contracts that generated significant cost-savings and helped to target very specific partners’ needs. The Project implemented a number of strategies to attract high-calibre experts by announcing the opportunities on the web and newspapers and disseminating through emails to the partners and indirect Project beneficiaries.

Local experts were engaged whenever possible instead of international consultants.

The Project demonstrated a high degree of flexibility in adapting to the changing environment. The Project quickly responded to changing Government priorities and implemented a range of activities to improve the quality and availability of regional statistics to support the Government’s decentralization and sectorial strategies. In conclusion of this section of the report, the evaluators would like to state that despite challenges, overall management of the Project, including financial management and management of partnership relationships was satisfactory and efficient. The Project had only a Manager and an assistant on board and the Project’s success could be attributed to their hard work and excellent organizational skills that ensured the achievement of all Project`s expected outcomes in difficult changing political and socio-economical circumstances.

5.4 Impact

Impact deals with whether the Project’s overall objectives have been achieved or are likely to be achieved. The evaluation of the Project impact is very wide in scope and complexity and the evaluation team went well beyond specific Project deliverables and explored whether the Project had contributed to changes in views and mindsets, policies, programs, awareness and behaviours.

The findings of the evaluation prove that the Project activities contributed to systemic positive changes in the field of statistics and evidence-based policy making in Moldova.As the Project document does not explicitly distinguish between outputs, outcomes and impact indicators, the evaluation team exercised its professional judgement in the analysis of specific areas of impact covered below:

Numerous statistical indicators designed with the Project support are routinely produced and most likely will continue to be produced and disseminated by the NBS after the Project completion that demonstrates NBS’ confidence about data methodologies and its ownership.


48

Statistical methodologies in many areas were aligned with the EU and international standards that facilitate cross-country comparisons and support Moldova’s EU integration processes. In 2010 the Republic of Moldova started the process of Negotiations of the EU-Republic of Moldova Association Agreement. Under the terms of the process, the national statistical systems should be aligned with the European norms and standards respecting the UN Fundamental Principles of Official Statistics and the European Statistics Code of Practice. The Project made a significant contribution into this process by bringing experts in specific areas of statistics to assess their compliance with the EU standards.

Capabilities of NBS in data collection and dissemination and responsiveness to users’ needs have significantly improved.The accuracy, quality, and in some instances timeliness of statistics have improved. A range of easily accessible statistical products has increased.The Project views data users as an integral part of the statistical system, which is crucial to improvement of collection, processing and dissemination of quality data and ensured that NBS always takes into account the users‘ perspectives in the production of statistical outputs. To institutionalize a users-centred focus of statistics, the Project organized and facilitated numerous consultations, users’ needs assessments and open information exchanges of NBS and data users in Government, academia, CSOs, donor agencies, and international organizations. As the evaluation of the Programme Promoting Gender Equality in National Development Policies and Programmes in Moldova found, such an extensive participatory process was implemented to revise and improve the publication Women and Men and improve quantitative data and elaboration of gender-sensitive statistical indicators.49

NBS and Government capacity to monitor Moldova’s progress toward the Millennium Development Goals has significantly improved so that the country’s progress towards MDGs is assessed relying on international methodologies and the data produced are reliable.

Some foundational elements of the evidence-based decision-making culture were established. The users and data producers themselves started to appreciate more the value of quality, timely and disaggregated statistics that generated demand for better statistics. Positive changes occurred in culture and practice in statistical and other evidence use by line ministries. Many interviewees and focus groups participants provided specific examples of statistics use in policy development and monitoring. The availability of sex-disaggregated statistics helped for instance the Ministry of Labour, Social Protection and Family to design better targeted strategies and policies.

The users’ levels of trust in statistics have improved dramatically and this can be to a large extent attributed to Project’s interventions. The first survey on users` satisfaction (2008) highlighted that “Even if the majority of the respondents appreciated NBS provided data as being at least satisfactory, only 34,7% of the users state that they trust NBS statistical data, 32,9% being sceptic regarding the data, 15% - sceptic towards the data from certain domains”. The second satisfaction survey (2013) found that “the level of trust toward the official statistics is quite high and about 90% (!) of the users expressed their confidence in the statistics gathered by NBS”. Particularly, 29,5% of users have mentioned that they trust pretty much the

49NivesMiošić-Lisjak, International consultant, and Gheorghe Caraseni, Local consultant, Final External Evaluation Report of the Programme Promoting Gender Equality in National Development Policies and Programmes in Moldova, 2007-2010, 2010.


49

statistics of the NBS, the majority of users (64,4%) have mentioned that they have a certain confidence in the data. As for the users who are sceptic regarding the data it decreased from 47,9% (in 2008) to the 3,7 %. It is obvious that the capacities of NBS were improved and the users acknowledged that the data quality, timeliness and availability have improved. The mass media representatives stated that they are mostly satisfied with the quality and quantity of the statistics provided by NBS, while the public officials, CSOs and international organizations require more disaggregated data. The surveys reveal, also that users are least satisfied with the statistics on health, social protection, tourism, culture, sport, national accounts, business and investments.

Although the Project’s contribution to the NBS website development was limited to shaping of its content, NBS greatly appreciated this support. The website became an essential source of statistics for increasing numbers of users. A wide range of diverse groups of users from Moldova and other countries are using the official website of the NBS (www.statistica.md) and the numbers are steadily increasing. Thus, in 2012 186,000 unique visitors from 165 countries accessed 1. 8 million pages. An important progress indicator is the fact that the NBS` website was about 5000 times accessed from mobile devices, which is 3 times (!) more than in 2011.The most popular topics accessed by users are: Population Prices, National Accounts and Statistics wage. The website is well connected with the social networks and blogs and the NBS` page on Facebook is getting constantly more subscribers, for instance in 2012 the total number of subscribers increased by 28% in comparison with 2011. The Databank "Statbank" http://statbank.statistica.md remains one of the most accessed data dissemination tools in 2012, allowing users to create individual tables from about 500 existing tables grouped into 24 statistical areas. The Databank has been accessed by 43,000 people from 110 countries, which represents 11.6% increase from 2011.

Statistics are more extensively reflected in Government’s Strategies, Programmes and policies, in particular in social and gender fields that were specifically targeted by the Project. The statistics are extensively used in such diverse Government documents as Government‘s General Action Plan 2009-2011, National Strategic Program on Demographic Security, the National Development Strategy, and the National Programme on Gender Equality forthe period 2010-2015 and policies on effects of aging. Social statistics are extensively used by a number of ministries: the Ministry of Labour, Social Protection and Family in its Programs, annual Social Reports and diverse social policies and analytical reports; the Ministry of Economy that develops Reports on poverty and policies impact and calculates Small Area Deprivation Index. It seems, however, that line ministries predominantly use the statistics for reporting purposes, especially with regard to international agreements and conventions and less for policy development and monitoring.

Although a number of high quality analytical publications were produced by the Project (e.g., social inclusion, gender violence against women, child labour) and these publications were widely disseminated, the actual impact of these activities remains under-examined.

Users’ satisfaction with the available NBS’ statistical products has improved.A range of data users has increased and includes more NGOs, academic institutions, journalists and students. Despite these positive changes, some politicians and decision-makers still lack confidence in statistics and tend to base decisions on their assumptions about the world and the prevailing ideology. This creates an environment where strategies and plans are devised and

http://www.statistica.md/


50

implemented without sufficient attention to empirical evidence and with weak monitoring and evaluation practices. This in turn further depresses the demand for data. To break this vicious circle, broad public awareness campaigns and targeted interventions equipping decision makers with the tools and guidance to interpret statistics could be implemented. This is particularly necessary to support the Government commitment to approximation to the EU statistical standards and improved use of statistics in evidence-based policymaking.

The capacities of partners that benefitted from the Project’s training opportunities have increased. The evaluation team could rely only on the anecdotal evidence gathered through the interviewees and focus groups and did not have systemic evidence to confirm this observation.

The evaluation team acknowledges that not all recommendations produced by the Project were accepted and implemented by NBS. For example, the Project supported a feasibility study regarding establishment of a Training Unit/Center that could improve in-house statistical literacy and develop data users’ skills and abilities within the NBS.50 The feasibility study, including scenarios on establishment of such a training unit was produced by the consulting company, and the report has been submitted for NBS consideration. As the political and policy environment and NBS and partners’ priorities change, it would be naïve to expect that all recommendations produced by the Project will be implemented. To maximize the Project impact, however, it is beneficial to monitor the status of Project recommendations, identify specific barriers that prevent the Government and other partners from implementing them and develop some corrective actions, if possible to increase the chances of their acceptance.

5.5 Sustainability

This part of the report assesses sustainability prospects at the level of national statistical system. It also assesses the level of political leverage and key stakeholders’ buy-in at the end of the Project, including necessary further support to strengthen the statistical system in Moldova. Sustainability deals with being strategic and looking to the future and assessing if key Projects elements will be in place and expand beyond the Project completion.

The evaluation team found that there is a substantive level of ownership of the Project within the NBS, at both management and staff levels who enhanced their skills and confidence to carry the reform process forward. The NBS management and staff were very clear in identifying their strengths and needs and were open to exploring new statistical tools, processes, methodologies and adopt successful international experiences. Sustainability of the Project, however, is tightly connected with the degree of political commitment to public administration reforms and recognition of the importance of statistics in policy making and monitoring.

The main evaluation findings regarding Project sustainability are listed below:

Numerous statistical indicators and methodologies that were developed by the Project are deeply institutionalized in the NBS. They are routinely produced and most likely will continue to

50 Terms of Reference Elaboration of a feasibility study for establishment and running of a Training Unit to improve statistical literacy and build capacities of data users


51

be produced by the NBS after the Project completion that demonstrates NBS’ confidence about data methodologies and ownership.

National ownership requirements have been substantially met at the highest level of Government and public officials, which has demonstrated strong support and commitment. There is also proof of increasing ownership among various line ministries and governmental agencies involved in the Project and buy in from data users and other stakeholders.

Due to the Project, statistical methodologies in many areas were aligned with the EU and international standards that facilitates cross-country comparisons and support Moldova’s EU integration processes. The Project made a significant contribution to this process by bringing experts in specific areas of statistics to assess their compliance with the EU standards. As NBS will have to harmonize eventually all its statistical methodologies with international and EU standards (European Statistical System), the successful practices developed by the Project will most likely be followed by NBS and replicated for other statistical areas.

The Project strengthened NBS capacities in the area of social and gender statistics. The evaluation team does not believe that the extensive social and gender surveys supported by the Project will be regularly produced by NBS without external support as it requires significant budget resources. These ad hoc surveys may not be continued by NBS unless they are identified as priority areas for policy interventions, which would make possible to plan in advance the necessary funds from the national budget or the budgets of beneficiary institutions.

NBS developed a comprehensive list of 41 needs for future development of national statistical system, while Central Public Administration institutions formulated 23 proposals. They partially reflect Project’s successes and demonstrate that partners understand the importance of statistics for data evidence-based policy-making and are willing to collaborate with development partners within the new RM-UN Partnership Framework 2013-2017.

The Project capacity building activities targeting both data producers and users made a significant impact but may not be sustainable in the long run as they did not become institutionalized within NBS or other key Government policy institutions. The only exception is an innovative Course on How to Use Statistics that was established at the Department of Journalism and Communication at the State University of Moldova and is included into a list of mandatory courses for its students. The Project trained public servants, journalists, NGOs and other partners and the evaluation team believes that they will continue using statistical data in their work and rely on knowledge and skills acquired through the training and other capacity building activities organized by the Project. With the risk of staff rotation/attrition, some of the Project’s achievements in capacity building will be unsustainable if not institutionalized.


52

There are a number of external factors beyond the Project control that positively contribute to its sustainability:

The Government EU accession plans will have to be supported by continuous improvement of statistics and their alignment with the EU standards.

The UN-Moldova Partnership Agreement identifies a number of reforms in public administration and confirms commitment of all partners to establish a modernized public administration system capable to effectively and efficiently develop, budget, implement and monitor evidence based policies in support of the country’s national priorities and European integration objectives. One of specific outcomes listed is “Ex- ante policy analysis and results based management principles mandatory for public policy development, ensuring results oriented, rights based and gender response implementation and monitoring with clear linkages to program based budgeting.”51 It is clear that these objectives cannot be realised if statistics production, dissemination and especially use do not improve.

The evaluators emphasize that there are a number of factors beyond the Project control that may negatively affect its sustainability:

o Political uncertainties that may undermine Government commitment to EU integration and promoting evidence-based decision-making practices.

o The NBS’ capacity to lead reforms of the national statistical system has to be continuously strengthened.

o Global economic crisis and lack of domestic structural and economic reforms may negatively affect the economic performance of Moldova and result in declining NBS budgets that will have a detrimental effect on Project sustainability.

Finally, given the low levels of understanding and importance given to statistics among policy makers and other stakeholders, the interventions in the area of statistics have to be continued to make a real impact on policy and culture of decision-making.

5.6 Human rights-based approach, equity and gender issues

51 United Nations – Republic of Moldova, Partnership Framework 2013 – 2017: Towards Unity in Action, Action Plan.

What do the NBS said?

The Project is very useful for us because it helped us to improve our knowledge, to get new experience, to develop tools, adjust methodologies etc -------------------- The project manager helped us a lot in ………………. ------------------- The experts provided by the Project helped us a lot. ------------------- Although there are metadata, the line ministries are not using them. ------------------- We have to focus more on using the data and supporting line ministries. ------------------ I think we have to focus more on implementation of the GIS (Geographical Info Systems) ----------------- We have to set up a Training Center ----------------- We still need capacity building support on IT technologies and set up one common data base


53

The evaluation team examined the extent to which the Project outcomes reflect and contribute to the improvement of availability and use of disaggregated data to capture the situation of the most vulnerable categories of population. Human Rights and Gender Equality remained a priority in the overall Project budget and implementation. The Project ensured that its activities facilitate the claims of rights-holders and the corresponding obligations of duty-bearers by providing better quality and disaggregated data to better identify and address the immediate, underlying and structural causes for not realizing such rights. The statistics produced due to Project interventions informed the legislative and policy work to ensure that institutional and policy roadblocks to realization of human rights for vulnerable groups and gender equity are eliminated.

The Project strengthened NBS’ capacity and supported data collection and analysis of evidences measuring the degree, scale, purpose and frequency of both, women and men, participation in social, cultural and economic life. The capacities of duty-bearers to fulfill their obligations have been strengthened through numerous activities, such as capacity building and awareness raising activities on how to use statistics in developing social inclusion policies and mainstream gender into all Government policies. As a side effect, the Project made a significant contribution towards the achievement of gender equality through making gender-disaggregated data and gender statistics available and accessible.52 Women and Men in RM. Analysis from a Territorial Perspective53 includes statistics produced by NBS and data from numerous administrative sources in disaggregated form and captures reality of women and men in all spheres of social and economic life. Violence against Women in the Family in Moldova is a first analytical study on violence against women based on official statistics that identifies the root causes, estimates the prevalence of different forms of violence as well as explores the multiple impacts of violence on victims. The metadata produced with the Project support include gender statistics54 and Moldova’s progress with regard to MDG3 on women empowerment indicators was captured as well.55Gender perspective is a cross-cutting aspect of the Project activities, under all its three components: data production, data dissemination and, in particular, data use.

The Project conducted a range of trainings to increase gender awareness and skills of using gender data in policy development among public servants. The trainings focused on skills of how to systematically monitor gender situations and address gender inclusiveness and identify priority areas for interventions to improve women’s situations through drafting and implementing specific gender strategies, action plans, or sector programs. Such diverse groups as staff from Social Assistance Sections from Rayon Councils;56 Gender Focal Points in line ministries and Policy Monitoring and Evaluation Units/Divisions in line-ministries benefitted. Despite increasing availability of empirical data and extensive trainings implemented, line ministries’ capacities in using gender disaggregated data for measuring, monitoring, and assessing gender disparities and developing the corresponding policies remain weak. A balanced gender representation was ensured on the Project team and the Steering Committee.

52It was one of the recommendations of the Gender Audit conducted for UNDP Moldova. 53http://www.statistica.md/pageview.php?l=ro&idc=263&id=2199 54http://www.statistica.md/pageview.php?l=ro&idc=430& 55http://www.statistica.md/pageview.php?l=ro&idc=433& 56http://www.un.md/key_doc_pub/un_magazines/2012/UN_Magazine2.pdf, pages 7-8

http://www.statistica.md/pageview.php?l=ro&idc=263&id=2199



http://www.un.md/key_doc_pub/un_magazines/2012/UN_Magazine2.pdf


54

5.7 Partnership strategy The Project’s success was to a large extent determined by the effectiveness of UNDP partnership strategy. The project represents one of the few Joint UN initiatives in the country and it demonstrated usefulness and high impact of such interventions based on joint efforts and synergy of UN partners. It responds to the needs of the Government and it scored well in terms of data collection and data dissemination. Each UN agency involved into the Project has its own areas of expertise and responsibility. The evaluation team found that effective partnership was ensured at both programmatic and funding levels where the roles of key partners were clearly delineated. UNDP and UN Women, for example, is jointly supporting the activities providing methodological assistance to implement ESSPROS/ European system of integrated social protection statistics in Moldova. UNFPA supported NBS in implementation of a range of advocacy interventions in preparations for population and housing census. UN Women supported major research on gender inequality in Moldova. ILO and UNDP funded the statistical survey of households on Labour force migration that was an ad-hoc module to LFS. ILO/IPEC and UNICEF supported Child activities Survey in 2009. UN Women, UNDP, UNICEF and UNFPA supported evidence generation/production and research on gender related topics, including time use by women and men, violence against women, situation of Roma population, etc.

Each of UN partners relied on their extensive international networks of expertise. UNDP, for instance, engaged UNDP Bratislava Regional Centre’s experts to improve statistics for monitoring human development and social inclusion. ILO extensively used its network of labour statistics experts. UNICEF brought its experts in the field of education statistics. NBS staff highly appreciated the support provided by ILO and UNDP experts.

As the evaluation team acknowledged in the relevance section, the Steering Committee established by the Project is an effective partnership mechanism linking the NBS, statistics users and international partners. Through annual review of Project priorities, the Steering Committee identified and revised Project’s priorities and activities that should be implemented by each UN system partners. The Steering Committee was to some extent performing the functions of Sector Council on Coordination of Technical Assistance in the Field of Statistics as the Council was not meeting for a long time.

The national partners are highly satisfied with the Project partnership strategies and they appreciate that they voices are heard and interests addressed. The UN Joint Project developed a comprehensive and well functioning coordination mechanism that works for all partners involved and the evaluation team acknowledges that the Project manager played an instrumental role in success of these coordination efforts.

As there are a few partners supporting NBS such as the IMF, Statistics Norway, the Project management ensured proper co-ordination of activities with other diverse partners to encourage synergy and avoid overlaps.

SUCCESSFUL PRACTICES AND LESSONS LEARNED

The evaluation team identifies the following successful practices in Project implementation:


55

In terms of programming, the most valuable lesson learned, that should also be viewed as a good practice example, is a strong partnership of all UN agencies that was critical to its success. The comprehensive nature of the joint Project that focuses on such diverse areas as institutional building, improving data collection, processing, use, capacity building, supporting the incorporation of data into the policy cycle and multiple beneficiaries could not effectively implemented by one UN agency. The Project adopted one UN approach and benefitted from UN agencies synergy where each UN partner contributed its expertise. It is an eloquent example when multi-stakeholder open consultative processes contribute to the practical application of good governance principles of openness and transparency, and participatory decision-making. Such processes can and should be replicated in similar initiatives in Moldova and elsewhere.

The Project was correct in addressing all key stages of the statistics process: development of indicators, enhancements of data collection and production processes, installation of quality control mechanisms, data production in user-friendly format and analysis of statistics and significant users’ capacity building to utilize statistics in policy development and monitoring.

The Project addressed both supply and demand for statistics at the same time. It takes time to build demand for high-quality data that can be stimulated by training, producer-user dialogue, and effective data dissemination.

The implementation of specialized surveys and ad hoc modules to existing surveys to explore social trends and other areas strengthened capacities among NBS to produce quality statistics in line with EU and international standards and learn new methodologies. Although these exercises could be seen as an additional burden on NBS’s staff, they demonstrated the abilities of NBS to produce high quality statistics and built staff confidence. In addition, the users realized that NBS could produce diverse statistics to inform their policy work and that they need to develop skills to use the data appropriately.

In terms of management, Project implementation has shown that working in uncertain political circumstances requires significant flexibility, adjustments to implementation plans to ensure both start-up and continuity of Project activities in conditions of political stale-mate. Therefore, the identified political risks in the Project documents are never to be taken lightly, as they represent the biggest possible challenge to successful Project implementation. In such circumstances, Project subject knowledgeable, creative and dedicated Project manager with partnership building skills are some of the keys to success, as proved by this particular Project.

In terms of efficiency, the fact that the Project Team was located on the premises of the NBS, reduced overheads, enabled permanent “face to face” contact, direct communication and continuous support to the key national counterpart at any needed situation.

The Project impact and sustainability was ensured through adopting of effective institution and capacity building interventions that built solid foundations for NBS to lead the reforms of the statistical system.


56

This is a consultative and participatory final evaluation with a strong learning component and the lessons learned are:

Fragmented, stand alone and uncoordinated capacity building activities do not result in sustainable improvements in statistical production, dissemination and use. A holistic and systematic approach to reforms is necessary as the improvements are needed in all parts of the system, including line ministries and other producers and users of statistics.

Reforms of statistics cycle are a time consuming process that requires continuous commitment of multiple stakeholders and a strong political will. The reforms of statistics should be linked to promotion of evidence-based decision making culture and should have strong support of the central public administration. NBS should assume more significant role in analysis of gathered and disaggregated data and producing reports, booklets and other resources to demonstrate the importance and multiple uses of statistics.

NBS has the necessary authority under the current legislation to strengthen capacities of line ministries in administrative data production and use, but central public administration support is necessary to ensure that this function is effectively performed is required. It is necessary to better coordinate the projects and supports targeting data production and use with NBS to promote synergies and avoid potential duplications.

The strong commitment and ownership by the NBS and close coordination of all stakeholders are critical to Project success.

The improvements of the national statistical system should have a clear focus on users. Users should be clearly identified and supported through diverse awareness raising and capacity building interventions to demonstrate through practical examples the benefits of quality statistics and how to use the data. It is insufficient to conduct seminars, roundtables and other ad-hoc activities to support users – the interventions should be systemic and comprehensive, with some practical hands-on exercises tailored to users’ data needs.

The biggest challenge of any interventions in the area of statistics is ensuring sustainability of reform efforts. It is not sufficient to ensure a broad partners’ buy in of reforms - strengthening capacity of key partners is needed to ensure sustainability.

The Project’s LogFrame should be regularly updated to reflect the changing Project circumstances. The main intended outputs, activities, results and objectively verifiable indicators should be adequately defined, appropriate and stated in measurable terms.

CONCLUSIONS

Despite many challenges due to political circumstances and small project staff, this complex UN Joint Project, with its multi-stakeholder and participatory approach, has achieved all expected outcomes and numerous unplanned results representing the added value of the Project. In the evaluators’ opinion, the key success factors were the joint efforts of the implementing partners, dedication and pro-active stance of the Project team and sensitivity


57

and high levels of engagement of NBS who provided outstanding support in Project implementation.

Key stakeholders, especially from the government and other international development organizations, NGOs, Academia and other data users generally view the Project as very relevant and successful in terms of reaching its outcomes, and they value UNDP extra efforts that were invested in order to surpass the challenges, to consolidate the efforts of all partners and to strengthen the statistical system, particularly the NBS and influence positively policy making processes.

The Project managed to build effective partnerships with stakeholders, involving key national players from the Government, line ministries, civil society organizations, think tanks, academia and mass media. The Project was also successful at the level of the governmental and UN community where it managed to mainstream statistic issues both into the governmental and UN priorities and strategic documents. Therefore, this UN Joint Project is an illustrative case study of a success story, on how effective and efficient a Project can perform (regardless of the challenges) when many partners put together their efforts in a certain area of cooperation to achieve synergy.

A high level of visibility has been created by the Project through good reporting and documentation in hard copy and online and in developing a solid body of information and knowledge for use and reference. This includes the various seminars, round tables, studies undertaken for informing Project, highly appreciated NBS` website, an impressive number of the informational materials produced and disseminated. As discussed throughout the report, the Project proved to be very flexible to the needs of the NBS, reacting on time, providing valuable and tangible support to NBS and involving high quality national and international experts. At the level of the achievements, the Joint UN Project scored well. The effectiveness of the Project at the level of components related to strengthening the national statistical mechanisms assessed as high, in the first place due to the wide, diverse and results oriented capacity building interventions and improving the evidence based policy making of different ministries for different types of reporting on national and international commitments of the Moldovan Government. Project-supported activities have positively impacted the capacities of the NBS to develop new statistical tools, more relevant indicators, better methodologies, while the multi-stakeholder approach had a positive effect on the collaboration between the NBS and line ministries and other users. Interventions developed and implemented in the scope of the Project have proved to be necessary to sensitize the decision-makers, international development organizations and the general public alike, so as to make sure that Project efforts and their effects reach a “fertile ground”. Given the low levels of understanding and importance given to statistical issues among the general population, the impressive results achieved by the Project should be further promoted and new development initiatives designed.


58

Overall, management of the Project, including financial management and management of partnership relationships was satisfactory and efficient, having ensured the achievement of all Project`s expected outcomes in difficult changing political and socio-economical circumstances.

RECOMMENDATIONS

Recommendations set forth in the evaluation report suggest possible strategic and follow-up activities for UNDP, NBS and other stakeholders from the governmental officials, line ministries, mass media, think tanks and civil society organizations in order to increase sustainability prospects of project achieved results, or/and to enhance their effectiveness. Some of them can be implemented by UN/DP within the scope of the follow-up initiative and, therefore will be reflected in a separate document - Concept Note, while other can be considered by all partners. The evaluation team has undertaken a brief diagnostic exercise to identify key political and economic conditions and other important contextual constraints, opportunities and key driving forces that are critical for the next project design, stakeholder buy-in and successful implementation. The following main contextual factors relevant to the future project have been identified: 1. Economic growth in Moldova may not be sustainable if external circumstances do not improve and domestic fiscal, structural and sectoral reforms are not implemented. Accordingly to the official information, Moldova returned to economic growth in 2010 and has achieved cumulative GDP growth of 15% during the period 2010-2012. This growth, however, has been jobless and driven mainly by remittances rather than capital inflows, exports or foreign direct investments. In 2012 the Moldovan economy experienced a sharp slowdown but is expected to pick up in 2013. Such external factors as the euro-zone crisis, the contraction of external demand, mainly in the EU to which roughly half of Moldovan exports goes, continuous volatility of food and energy prices, and climate risks are the key external challenges to Moldova’s economic growth. In addition to external factors negatively affecting prospects for economic growth, the Government made limited progress in implementing deep fiscal, structural and public administration reforms that are needed to build solid foundations for human development. Such economic uncertainties negatively affect the Government’s revenue generation capacity. As a result, the budget funding of NBS is insufficient to cover systemic reforms such as procurement of modern IT infrastructure, ensure sustainability of some donors’ interventions and it limits NBS capacity to support institutionalization of evidence-based decision making across the central agencies, line ministries and local authorities. 2. Social exclusion is a multifaceted phenomenon that has to be routinely monitored to ensure proper policy responses. Accordingly to the official data, in 2010 poverty fell to 21.9% down from 30.2% in 2006 that can be attributed mostly to rising remittances, increased social assistance and rising incomes from Moldovan economy mainly from agricultural activity. Although the 2010 improvements were mostly driven by declines in rural poverty, it remains in many ways a “rural” phenomenon as 80% of country’s poor reside in rural areas. Poor rural residents tend to have lower levels of educational attainment and more children in their families. Rural population also suffers from much worse access to other basic services, such as water and sanitation (90 % of


59

population without access to public sewage systems are rural residents). The complexity of social inclusion dynamics in Moldova requires routine monitoring to ensure prompt and effective evidenced based policy responses. 3. Migration, despite its economic benefits, remains a big challenge for human development of Moldova. Remittances constitute 22 per cent GDP (down from 32%-peak in 2007) and play important role in keeping afloat rural households. The number of migrating Moldovan workers continues to grow and this results in labour shortages and social challenges, such as children living in families where one or both parents work abroad. Multiple interviewees with the project stakeholders indicated that the current estimates of country’s resident population are inaccurate as the size of Moldova’s non-resident population is unknown. Although the administrative data about vital events are reliable, the 2004 Census data that had some quality issues mostly due to non-responses and unreliable migration data make demographic statistics often unreliable, especially when disaggregated to the levels of localities.57 These data limitations hamper local planning and development and are obstacles to effective decentralization. 4. Political instability as the Government including the Prime Minister and the Chair of Parliament were recently dismissed.58 The Parliamentary Elections held in April 2009 marked an end to eight-year Communists’ rule occurred amidst allegations of rigged elections, street unrest and violent crackdown by police, which later came to epitomize the widespread human rights’ violations by the previous regime. The new Government was extremely effective in rebuilding relations with key external partners and committing to the European integration path. Since 2010 a series of key reforms have been started, such as Justice and Anticorruption Reforms, Public Administration Reform and Decentralization etc. However, the recent political events put under the question mark the prospective of Association Agreement59 and threaten future political association and economic integration between the EU and Moldova. The political stability was short-lived and unless the three governing parties renegotiate a new Coalition Agreement, Moldova would have the anticipated elections. In the opinion of the evaluators, the political instability may not be conducive to major public administration reforms, including major political support of reforms of the National Statistical System and institutionalization of evidence-based policy making.

The important decisions regarding the next project should be made in response to pressing stakeholders’ needs and demands, but be supportive of the Government of Moldova priorities and key objectives outlined in such strategic documents as Towards Unity in Action, United Nations – Republic of Moldova Partnership Framework 2013-2017.

The following guiding principles were adopted in developing recommendations for a new UN Joint Project in the area of statistics and evidence-based policy making:

a) Policy and regulatory instruments of promoting use of evidence in decision-making are unlikely to work in unstable political climate of Moldova. In ideal circumstances, it would be beneficial to integrate the next project in the area of statistics into the broader public administration

57 Interviewees with the Demographics section management and staff of NBS 58 The Moldovan Government incl Prime Minister VladFilat was dismissed in March 2013. The Parliament passed a vote of no confidence against the Government and one month later (April 2013) the Chair of Parliament, Maria Lupu was dismissed. 59 Initially planned for September 2013 in Vilnius, Lithuania.


60

reforms and strengthen the capacity of all national and regional actors in using statistics in the policy process, but it may be problematic in the current unstable political climate in Moldova. There is a high risk of reversibility of policy actions in the area of public administration reforms. In light of these political fragile realities, it is advisable to focus the next joint project on “soft” instruments of promoting evidence-based decision making, such as: capacity building mainly through trainings and consultancy of multiple and various statistics users, improving of quality of administrative data and alignment of methodologies used by NBS and line ministries to EU/international requirements, and expanding a range of data dissemination instruments. In this context, it is advisable to position the next project within the NBS that is less susceptible to changes in the political leadership.

b) The Government’s commitment to EU accession is a strong key driving force and a leverage point to strengthen statistics and promote evidence-based policy making. In light of pro-EU orientation of Moldova, the next Joint UN project in the areas of statistics should be well coordinated with the EC Delegation and consulted with some think tanks and line ministries to ensure that future activities are supported by civil society and main users and aligned with EU approaches.

c) The EC Commission is interested in launching a major project in the area of regional development that will have a significant regional statistics component. In this context, it is recommendable and it is important to avoid duplication and seek complementarity for a next UN Joint project. One of the components of the EU-funded project is focusing on improving regional statistics as a basis for improved, evidence-based policy planning, programming, monitoring and evaluation.The main goal of this component is to achieve markedly improved regional statistics – consistent as far as possible with Eurostat and international (OECD) standards. It includes activities to review regional statistical indicators currently produced by NBS and other authorities, identify required statistics and support to NBS on compilation of a complete list of regional statistics indicators across all main sectors such as economic, social, environmental, transport, health, education, employment and labour market, energy, etc. The systems to collect, process and make available regional statistics will be developed through capacity building and methodological advice to NBS. It is important to underline that the component will target the users as well and develop systems and procedures for using regional statistics for regional development planning, monitoring and evaluation purposes.60 When this summative evaluation was conducted, the evaluation team was unable to obtain more details on the regional statistics component of the forthcoming EC regional development project.

d) The overarching goals of the next project in the area of statistics should be to contribute to creating an appropriate environment of evidence based policy making conducive to alleviating social exclusion and promoting sustainable human development in Moldova. Statistics should not be produced solely to capture Moldova’s progress with regard to meeting its international commitments, but should be deeply embedded into a policy cycle. The policy cycle is a set of logical steps that is normally initiated by a political decision that is followed by more in-depth analysis of the issue, challenges and potential solutions. The public service develops options for Minister’s consideration by taking into account the objective evidence, statistics and administrative data, Government political commitment, positions of key stakeholders, public and

60EC Action Fiche for the Republic of Moldova, Pilot Regional Development Programmes (PRDPs), 2013


61

media influences and other factors.61 Following extensive consultations with partners, other line ministries and stakeholders, policy options are finalized and presented for Minister and Cabinet of Ministers review and decision making. Once a final policy decision is made, policy is being implemented and is monitored through regular performance measurement that relies on statistical and administrative data and adjusted, if necessary.62 Different types of users should use official statistics and administrative data at all stages of this process to develop the most relevant, effective and efficient policies that will have the maximum impact and adjust policies relying on the data, if necessary. Once the users start to fully appreciate the value of quality statistics, it will stimulate new demand for high-quality data and will provide additional impetus for sustainability of any efforts strengthening capacities of data producers.

After consultation a wide range of project stakeholders, the evaluation team is recommending that the quality, availability and accessibility of data should be improved, more disaggregated data should become available and it should be actually used for policy development, planning, budgeting and monitoring. Although the project scored well on under the data production and dissemination components, the data production, dissemination and use stages should be further targeted. However, the evaluation revealed that the focus should be shifted to the data use stage to maximize the project impact, as the use and demand for statistics in evidence based policy making is inconsistent among users. Continued UN support will be necessary not only to improve data collection and to promote its use in evidence-based policy making, but also to create internal demand for quality and timely statistics among a wide range of partners that can include the media, NGOs, think tanks, local governments and other actors such as private companies or international development entities.

c) It is necessary to maintain a systemic approach to statistical capacity building and maintain collaboration among the relevant UN Agencies and other development partners and bilateral donors and create necessary synergy. The supports should not be based on ad-hoc requests for specific statistics and a strategic approach addressing both capacity needs of the NBS and Government policy objectives should be pursued. In addition to a Joint UN project, NBS has been supported through a number of projects funded by the EU, Norway, Romania, France, Turkey and Sweden. Therefore, it is recommended that the future UN Joint project to continue close collaboration with all partners involved and, as mentioned already, to avoid duplication and optimize resource allocation. The following recommendations regarding key priorities for a new project in the area of statistics and evidence-based policymaking should be seen as an agenda of possible topics for discussions between UN Agencies, NBS and other relevant Government institutions, partners and donors. Some recommendations require significant funding commitments and may not be realistic to accomplish in the short term being suitable for a long term perspective.

8.1 Recommendations for NBS

61Siobhan Campbell, Siobhan Benita, Elizabeth Coates, Phil Davies and Gemma Penn. 2007. Analysis for policy: evidence-based

policy in practice. Government Social Research Unit. HM Treasury. United 62For more on the role of ministries in the policy cycle, see SIGMA. 2007. The Role of Ministries in the Policy System: Policy

Development, Monitoring and Evaluation.SIGMA Paper no. 39


62

The project made a significant progress in improving statistical infrastructure of NBS, which is a combination of technical, financial, informational and human resources that constitute a basis of any modern statistical system and are essential for its successful operation. The evaluation revealed that the NBS is increasingly called upon to provide more comprehensive, sophisticated and disaggregated statistics. It goes without saying that the needs, demands and expectations from users are pressuring NBS to provide more source information, better access to disaggregated statistics and tools, and more analysis. In conditions of prevailing overall budget constraints, the availability of quality and timely data will enable the Government to better prioritize, develop more effective policies and focus public resources on the country’s main priorities.

To deliver on these expectations, the NBS along with its main development partners should conduct a diagnostic assessment and develop a long-term Strategy of National Statistics Development for the next five years to replace the currently outdated NBS strategies.63 The development of the Strategy will require extensive in-country consultations with all stakeholders including line ministries, think tanks, NGOs, academia and other users. In addition to outlining specific sequential steps to approach core Eurostat standards in data production, the Strategy should outline specific steps that will be undertaken to strengthen capacities of data users. The recommendations provided below reflect some core areas that can be covered in the Strategy and may be included in a new joint UN Project.

8.1.1. NBS should seek international partners’ funding and advisory support to introduce a coherent IT system and optimize structure and capacities of regional offices of NBS

Many individuals interviewed through the course of this evaluation expressed strong interest and need in better quality, timelier, disaggregated and regional and local level statistics. Despite significant progress made in improving quality, methodologies and expanded range of indicators, the NBS cannot fully meet these increased expectations, partially because it lacks a modern nation-wide IT system.

The introduction of a comprehensive IT infrastructure based on modern communication and information technologies for data collection, validation and processing will improve cost-effectiveness of data production, reliability and timeliness of statistical data. As mentioned some of the interviewed stakeholders, NBS is using diverse and not always compatible databases and it affects quality and compatibility of data. The IT infrastructure could be built on an international standard such as United Nations Economic Commission for Europe (UNECE), Generic Statistical Business Process Model or others recommended by the Eurostat.64 Equipment installation and software development should be accompanied by training of the regional NBS staff. The introduction of the IT system will improve efficiency of data collection, align used definitions and methodologies and finally improve data quality.

63 The Government-approved Strategy of National Statistics Development for the period 2009-2011 and Plan of Actions of National Statistics Development for the period 2008-2011 are available at the NBS website at http://www.statistica.md/pageview.php?l=en&idc=152& 64See, for instance, UNECE Secretariat, Generic Statistical Business Process Model, 2009, http://www1.unece.org/stat/platform/download/attachments/8683538/GSBPM+Final.pdf?version=1


63

It is unlikely that NBS would have sufficient budget resources to fund the installation of the IT system and this gap could be filled by concerted efforts and financing of multiple UN Agencies and other partners.

In the opinion of the evaluators, the installation of a modern IT infrastructure should be aligned with the reforms of the territorial units of NBS. NBS should reduce the number of territorial units and improve their staff capacities in using IT, implementing quality control systems and conducting surveys. To provide the necessary statistics to support decentralization processes, it is advisable for NBS to introduce a spatial Geographical Information System (GIS) for providing maps for survey field work and to provide geographically-disaggregated and referenced statistical data. In the long-run, the GIS will allow linking various administrative and statistical data sources and promote the application of standardized concepts, definitions, and methods that comply with NBS standards across all line ministries that collect administrative data. The implementation of this recommendation will help also to avoid duplication of responsibilities for data collection between NBS and line ministries and this advantage should not be underestimated. GIS can also facilitate the dissemination of geographically referred data.

8.1.2 NBS may identify a limited number of administrative data collected by other bodies as “official statistics” and strengthen capacities of these bodies to produce quality and reliable data

Under the Law on Official Statistics, NBS is a coordinator of the Statistical System in Moldova. NBS prepares the programmes of statistical works, develops and decides on the methodology used for the production of official statistics. Line ministries and agencies collect diverse administrative data and it is beneficial to clearly identify those important data elements that could be considered as “official statistics” data. The concepts, definitions, and methods of data collection for these specific indicators will be defined jointly by NBS and respective line ministries/agencies. The identified administrative data will be checked by NBS experts and considered as statistical data. It will also help to avoid duplication of responsibilities for data collection between NBS and line ministries/agencies and ensure cost-efficiencies. NBS has already very positive experience of collaboration with the national customs office where the administrative data collected is transferred to the NBS, checked and considered as statistical data.65 NBS may seek the international partners’ support, if necessary to implement this recommendation. The next logical step for NBS would be to collaborate with the Ministry of Education that, with the World Bank’s support, will be transitioning from its current imperfect and patchy school mapping database to a consolidated Education Management Information System (EMIS) that links school-based data with financial data and the country's student assessment system. NBS in collaboration with the Ministry of Education and other relevant partners may identify some core administrative data elements that can be considered as statistical data and ensure that data collection and storing processes comply with the NBS standards. Again, it will avoid duplication and NBS will not have to collect the same data. It is expected that the problems of interpretation of data when statistical data are different from administrative data on the same indicators will be eliminated, as well.

65 According to NBS representatives, this project was supported by TACIS.


64

8.1.3 NBS can explore and implement a range of measures to strengthen society’s confidence in statistics

A reliable and trusted statistical system is the key in recreating trust in the state and establishing an

accountability link between the state, citizens, and private sector. The evaluation team learned that even some highly ranked Government officials in Moldova are sceptical of the quality of statistics. Policy makers in Moldova are in position to lead broad societal changes and promote human development but they have to trust statistics and be able to use it to inform their policy decisions. They often do not have sufficient awareness, knowledge and understanding of the importance of statistics and do not have sufficient skills to use it. When decision makers are not confident and unable to interpret official statistics, they tend to base decisions on their ideology and personal views and perceptions. This creates an environment where policies and programs are developed and implemented without attention to empirical evidence and with weak monitoring and evaluation systems and practices. This in turn further depresses the demand for data.

According to a recent Eurostat evaluation of statistical system in Moldova, “The principles of professional independence, objectivity and impartiality, and statistical confidentiality are kept in practice by NBS”.66 The principles of professional independence and statistical confidentiality that are fundamental to building public trust in statistics are in place and are important prerequisites that facilitate the long term trust building process and, finally contribute to mainstreaming the evidenced based policy making culture.

As mentioned, both NBS and its partners have to strengthen public trust in statistics by addressing both supply and demand simultaneously. The data provided by NBS should be relevant, timely and easy to interpret and use. The role of NBS should gradually change from the basic tasks of collecting statistics and disseminating it to assisting users to make appropriate use of the data. This means providing more explanations and interpretation of identified key trends. The evaluation team learned from multiple interviewees that the Statistical Council that had been established under the Law on Official Statistics to play advisory function to the National Statistical System have not been meeting for extended period of time. As the Council represents multiple partners, including users, it is advisable for NBS to re-activate the Council and involve it into development of strategic plans. It will improve responsiveness of NBS to users’ needs, consolidate the cooperation between NBS and other relevant actors, increase its public image and strengthen partners’ confidence in the NBS.

66 Vera Herrmann, Claudia Junker, Bronislava Kaminskiene, Günter Kopsch, Jason Schachter, Adapted Global Assessment of the National Statistical System of Moldova, 2012

NBS is very well aware that there is a need to create a statistical culture. It is a step forward to be made by NBS in order to meet the requirements of statistical users. It is NBS that will explain the statistical approach, what is the statistical methodology, and what is the message behind the statistical figures.

Interview with Daniela Stefanescu, UNDP international consultant, Director of the Department of European Affairs and International Cooperation of the National Institute of Statistics of Romania: Moldovan statistics is making good progress


65

It is advisable for NBS to seek development partners’ support and introduce quality management system by implementing the Plan for the Development of Quality67 that is based on the principles of Total Quality Management (TQM). The Plan contains the steps and actions to be undertaken for: (i) the organisation and management of quality; (ii) continuous improvement of the quality of statistics and integration of the quality management concepts and principles into the National Statistical System of the Republic of Moldova; (iii) Human Resource Management; and (iv) ensuring the data-user orientation of the statistics.

The statistical information should be offered in different formats and appeal to such diverse audiences as academics, students, government officials, parliamentarians, public servants, media and NGOs and be easy to understand and use. NBS can conduct a comprehensive analysis of users according to their functions and data requirements so that the needs of such diverse groups as Government, industry, academia, media, international organizations, and the community at large are identified so that NBS’ products can be customized accordingly.

NBS should be very open about limitations in methodologies, data collection practices, definitions and data. The reference metadata that have been produced and made available on the NBS website is a step in a right direction.68 Clear thought should be given by NBS to what knowledge should be translated and to which audience, keeping in mind how the knowledge could be used. Some examples include briefs on such issues as employment, social inclusion, trade and others of interest to the public. Short easy-to-follow brochures for broad public dissemination explaining how to interpret statistics can be developed and widely disseminated. Presentations and round-tables on some key statistics can be conducted by NBS and target decision makers, media and NGOs. Media conferences should be held NBS frequently.

Particular focus can be made by NBS on decision makers and key opinion leaders who are in position to influence and lead broad societal changes. The information gathered for them should meet their needs and expectations, provide easy to understand interpretation of statistics and be short and applicable. It is far not sufficient to get a general confirmation from decision makers that they value evidence. They should appreciate the importance of statistics by learning how to use it and NBS` publications, presentations, practical exercises and other activities may build decision makers’ confidence in statistics. The focus on data users should be more explicit and they should be equipped with the necessary tools for basing decisions on facts.

As for the data dissemination, NBS can expand its use of social media and post key facts on the popular platforms. Presentations of its core products such as HBS, studies on social exclusion and others could be made in the webinar format and posted on the NBS website and NBS` page on Facebook. The user satisfaction online surveys are very important because they identify the trends, needs, preferences, achievements etc and, therefore have to be continued and their results widely disseminated, for instance posted on the NBS website.69

67Plan of the National Statistical System of Republic of Moldova for the Development of Quality in Statistics, 2009 68Reference metadata are available in Romanian on http://www.statistica.md/pageview.php?l=ro&idc=402&id=2869 and http://www.statistica.md/pageview.php?l=ro&idc=433&. The structure of the developed metadata by statistical sectors and statistical surveys complies with the metadata format used by Eurostat. 69Public opinion poll concerning the data user’s satisfaction with available statistical data for the NBS, available in Romanian and English on http://www.statistica.md/public/files/despre/evaluare_opinii/Raport_OIKOS_eng.pdf. Results of online data users surveys are available on http://www.statistica.md/polls.php?l=en

http://www.statistica.md/pageview.php?l=ro&idc=402&id=2869


http://www.statistica.md/public/files/despre/evaluare_opinii/Raport_OIKOS_eng.pdf

http://www.statistica.md/polls.php?l=en


66

8.1.4 Establish an NBS Education and Training Center

It is evident that the project has been highly successful in strengthening the statistical system of Moldova reaching appreciable results explained above. However, the evaluation revealed that the capacities of key users are still weak. It is insufficient to strengthen capacities of data producers, to ensure that statistics positively contribute to human development in Moldova; the focus should be as broad as possible and target all actors of the system, including other data producers, respondents, data compilers, and data users. The evaluators are confident that the NBS has sufficient capacities and is well positioned to provide training and capacity building supports to these diverse audiences.

Statistical capacity building of all partners requires integrated and coordinated support that could take a long-term commitment. It is advisable for NBS, in partnership with UN agencies and other relevant stakeholders, to establish its own Education and Training Center that will perform at least three core functions explained bellow.

First, the Center will address internal NBS capacity building and HR development needs, focusing on strengthening training policies, programs and capacity, especially of NBS’ regional units. Regular support and training needs assessments can be conducted to accurately define gaps between existing staff competencies and the required competency level to fulfill the designated tasks. Training policies and programs will become an integral part of the larger HR management scheme, and geared towards supporting NBS corporate needs. The Centre will manage and provide a full course of essential and relevant management and technical training required by NBS managers and staff at all levels. The tailored trainings can be provided on statistical techniques and methodologies in different aspects of data collection, estimation techniques, data analysis and survey management, and other technical areas of official statistics; report writing, data presentation, and dissemination. The Centre will help to upgrade the knowledge, competencies, and skills of NBS staff to enable them to become experts with experience and skills comparable to Eurostat and other international standards. Training will include also seminars abroad and in the country, consultancies, and on the job training. The consultants strongly recommend the Centre to focus on increasing NBS’ capacities to undertake qualitative analysis and interpretation of produced and disseminated data. The NBS could set up a pool of trainers and consultants including international experts as well. A Training of Trainers could be helpful for training skills development.

Second, the Center will support diverse users, such as line ministries and LPA in understanding, interpreting and applying the data for policy development, monitoring and analysis. The activities will go beyond traditional presentations, seminars, roundtables and include comprehensive interactive training modules on how to understand and use statistics at all stages of the policy cycle.70 Line ministries and other users may designate selected staff to take these training modules that could conclude with tests and assignments building practical skills to use statistics. For instance, the training modules may demonstrate how to use statistics and other relevant data to inform ex-ante policy assessments71 and measure (ex-post) the impact of policies and programs.

70A quite similar entity is the National Institute of Justice which is providing educational courses and specific trainings in the justice area for judges and persecutors (Please see www.inj.md). 71An ex-ante policy assessment vis-à-vis the vulnerable groups is a multi-dimensional systematic approach that identifies how proposed policy measures may influence the vulnerable groups and assesses the impact andprobability of these influences. The ex-ante policy assessment provides the evidence (e.g., distributional effects of policy reforms

http://www.inj.md/


67

The training sessions should be tailored to the needs of the users i.e. very practical and demonstrate on specific examples through exercises, case studies, simulations and specific examples on how statistics can improve the quality of Government and public decisions and their effectiveness.

Third, the Centre will support universities and colleges in integrating topics on how to read and use statistical information into relevant courses and/or developing stand alone courses on data use. The Centre may develop generic lectures on statistics and their use that can be used by such diverse departments as Economics, Public administration, Political Science, Journalism and others. The Centre may identify Academics from Moldova72 who would like to reflect multiple use of statistics in their courses and deliver one or two day seminars to deepen their knowledge of statistics. In addition, the Centre may develop a series of webinars on how to use statistics and make them publicly available. This self-directed learning instrument could be a proactive approach and an effective training and development method to reach a large number of students, public servants, NGOs and other stakeholders.

8.2 Recommendations for UN partners:

8.2.1 Identify one or two priority areas for the next project of critical importance to sustainable human development in Moldova.

The project has a long experience in statistical capacity building, but in the evaluators` opinion a more systematic and focused approach has to be adopted for the next project. The UN partners have to prioritize areas that would have the most significant impact on human development in Moldova and reflect key Government priorities, international commitments and mandates of respective UN Agencies. The main outcome of these initiatives will be to develop a range of indicators in areas critical to human development, ensure that NBS produces them periodically and the users know how and extensively use them in their respective sectoral policies and programs. Analytical policy documents providing feasible policy options and considerations that would rely on solid statistical information will be produced. One of the possible approaches to identify priorities is to have the members of the project Steering Committee convene to discuss a plan of actions and identify what can be realistically accomplished in two or three years.

One potential area worth exploring by UN partners is sustainable human development. What is the impact of environmental changes on people’s lives and livelihoods? Which social groups and communities are most vulnerable to environmental changes? How to design and implement policies that would address both poverty reduction and social exclusion and environmental changes? To address these core questions, the NBS and relevant line ministries and agencies (e.g., Ministry of Environment, Ministry of Agriculture and Processing Industtry) should have the necessary data in their disposal. Environment statistics describe the qualitative and quantitative aspects of the state and changes of the environment, and their interaction with human activities and natural events. Environment statistics are integrative; they measure human activities and natural events that affect the environment; and they monitor the effects and impacts on the environment, and the social responses to environmental impacts. The environmental statistics can

on the well-being or welfare of the vulnerable groups) to make informed decisions to choose the most effective policies supporting the vulnerable groups. 72For instance, the Public Administration Academy


68

be enhanced by tracking of emissions and the socio-environmental costs, for example. Using the analytical framework of sustainable human development, NBS with the partners’ support may develop corresponding targets and indicators and potentially come up with a green development index that will capture realities of Moldova. Environmental NGOs could be involved as important stakeholders and subject knowledge holders. UNPF could be used as a starting point of discussion. For instance, the UN partners’ interventions in the area of statistics can support effective environmental management and help in ensuring that public policies, plans and normative acts at central and local level integrate environment and/or undergo strategic environmental assessment (UNPF pillar 3: Environment, Climate Change and Disaster Risk Management). The interventions in the area of statistics can help also to develop and implement disaster risk management strategies.

Another area worth exploring is social inclusion. The project may reinforce its social exclusion study and reflect the current status of socially excluded groups. It may inform development of National Action Plan (NAP) against poverty and social exclusion that could be aligned in principle with the common EU guidelines in the area of social exclusion. Such areas as income poverty, long-term unemployment, health and lifelong learning and others could be covered in depth. Laeken indicators that were endorsed at the Laeken European Council in 2001 to monitor progress of EU states in the area of social inclusion could be calculated as well. This exercise will help to generate a list of leading indicators that will be frequently and routinely produced by NBS to inform policies in the area of social inclusion. The line ministries should build their capacity to use these indicators in the policy cycle. UNPF could be used as a starting point of discussion. The UN partners’ interventions in the area of statistics can help to monitor the effectiveness of Government interventions in addressing outcome 2.4 of UNPF People enjoy equitable access to an improved social protection system. In particular, the improved statistics could monitor if right holders from vulnerable or socially excluded groups have increased access to services necessary for realization of social rights.

8.2.2 Strengthen NBS capacity to monitor implementation of the UN-Republic of Moldova Partnership Framework 2013-2017

The United Nations – Moldova Partnership Framework (UNPF) “Towards Unity in Action” has been developed through consultation with the United Nations Country Team (UNCT) in Moldova and Non-Resident Agencies, the Government of Moldova, civil society and other stakeholders.73 Under Outcome 1.1 Institutional Strengthening: Increased transparency, accountability and efficiency of central and local public authorities to bring the institutional and operational framework of the public administration in line with European standards, it specifically addresses the issue of strengthening national capacity to build result-based monitoring and evaluation systems, and to collect, analyze and use data, as essential components of the governance structure and provide critical information, empowering policymakers to take better informed. In consultants’ view, the objectives of strengthening statistics and promoting evidence-based decision-making equally apply to all pillars of the UNPF as human development, Public Administration Reform and social inclusion, as well as

73Towards Unity in Action, United Nations – Republic of Moldova Partnership Framework 2013-2017 and Towards Unity in Action, United Nations – Republic of Moldova Partnership Framework 2013-2017, Action Plan


69

advances in environmental management and disaster risk management cannot be achieved, if statistics and users’ skills in using data in policy development, implementation and monitoring do not improve.

The heightened focus of the Government and other stakeholders on evidence in decision-making has increased the demand for useful, high quality, and timely information. UN partners can support NBS and strengthen its capacity to produce statistics identified in the UNPF, as well as support other data producers such as Ministry of Health, Ministry of Economy, Ministry of Education, Ministry of Labor, Social Protection and Family. The following sequence of steps can be undertaken by the UN partners:

1. Identify the most pressing data needs relying on the information available from UN partners, Government, NGOs through the lens of UNPF, Government priorities and capacity/willingness to incorporate in policy cycle;

2. Create sectoral groups with NBS involvement to improve quality of administrative data and ensure that the data collected under the UNPF are used in the policy cycle;

3. Conduct roundtables and other forms of consultations with stakeholders to explore how new/improved data collected for the UNPF could be used by a broad range of stakeholders.

8.3 Recommendations for UN partners on the next project design and implementation modalities

In terms of project design, it is recommended to strengthen the focus of the next project on a narrow set of outcomes and clearly identify expected results in the project document. It is beneficial for the next project to update annually or more frequently, as necessary, the Log Frame that could help to make the activities, performance measures, outputs and outcomes more current, realistic, sustainable and better focused, as well as better clarify the linkages between the inputs, outputs, means of verification, expected outcomes and ideally impact.

The consultants mentioned already their concerns about the way the outcome and output indicators are framed. It would be beneficial during the planning process to develop the results chain in a more realistic way that might conceive of results of outcomes in various stages, such as immediate, intermediate and longer term.

Inputs are what UN system provides in terms of human, financial and informational resources. Outputs are what UN system delivers and these specific products/results and services are mostly under the control of the project. Outcomes are changes in the ways Government and other users or counterparts operate that are influenced in part by the outputs delivered by the project. Impact is the overall and long-term effect of the project - what stakeholders achieve in terms of bringing about changes that could be partly attributed to UN efforts as many factors affect the impact.74

The project should identify effective output and outcome indicators. The ability to track project progress and learn lessons relies on the selection of indicators. The definition and periodic revision of indicators helps in clarifying and making more realistic the outcomes the project would like to

74See, for instance, UNDP, Handbook on Planning Monitoring and Evaluating for Development Results, 2009; UNDP, Outcome-Level Evaluation: A Companion Guide to the Handbook on Planning Monitoring and Evaluating for Development Results for Programme Units and Evaluators, December 2011; UNICEF, UNICEF-Adapted UNEG Evaluation Reports Standards, July 2010


70

achieve. The consultants strongly recommend to have a more extensive range of objectively verifiable indicators and to collect evidence pertaining to the status of all project outputs and outcomes.

Some examples of outcome indicators include: user satisfaction rates have increase by at least 60% by the specific date; targeted statistical products are easily accessible in relevant media, with metadata and interpretation of findings; internationally accepted statistical techniques in collection are applied by the NBS, and statistical outputs are released in a timely manner in accordance with internationally accepted frequency and timeframes.

An essential element of the managing for results framework is the need to review and adjust measures and expectations over time as more experience and understanding are acquired. It is unrealistic to expect that UN partners would be able to identify a perfect set of results, activities and objectively verifiable indicators that will last for the whole new joint project duration. As the demands for project’s supports may change, the project should be able to quickly respond to these requirements and opportunities. Plan of project activities, including Project Organization Structure, could be reviewed twice a year to improve project responsiveness.

In terms of project implementation, it is strongly recommended to maintain effective coordination of activities with other multilateral and bilateral development agencies to avoid duplication and inefficient use of funds. The Project Steering Committee that proved to be effective in overseeing, monitoring the Project implementation and reviewing its priorities should be maintained. As multiple users and partners will be involved in the next project, it is advisable to conduct regular consultations with them to ensure project relevance and stakeholders’ commitment and support. The meetings may also help in supporting inter-ministerial coordination and resolving any issues.

In the course of implementation the Project gained a reputation of being able to implement a very extensive range of activities and meet multiple deliverables. As a result, the demands for supports from NBS’ departments increased significantly. When the evaluation was underway it became clear that the Manager was overburdened, as she had to oversee multiple activities. To resolve this issue, a number of measures can be considered. A full time position of a project associate responsible for NBS Education and Training Centre can be established.


71

ANNEXES

9.1 Questions used during semi-structured interviews that were tailored to each

target audience Relevance Relevance deals with the appropriateness of the Project design to the needs of reforms in the field of statistics and evidence-based policy making in Moldova. The consultants will examine the degree to which the overall purpose of the Project remains valid and pertinent in the current context and can provide basis for further statistics development in the country. More specific questions that will be asked include:

To what extent does the Project respond to the needs of reforms in the field of statistics and evidence-based policy making?

To what extent are the Project design and its objectives relevant vis-à-vis national policies and strategies?

How and why were the Project areas of interventions selected?

Was the Project based on a needs assessment? What are its findings?

Did the priorities of reforms change since 2007 when the Project was designed? Did the Project adjust its activities to reflect these changes, if necessary?

What was the mechanism of adjusting the Project focus and priorities? How inclusive was it? Did it work well?

What is the current typical policy cycle in Moldova and what is the role of statistical information in it?

Is there a requirement to rely on accurate statistics in cabinet submissions and other policy decision documents?

What is the role of statistical data in healthcare, education and social programs/services provision? Do the central and local authorities rely on statistical information in their programmatic decisions? Do the better data help to maximize the effectiveness, targeting and efficiency of Government policies and programs?

What is the level of Government’s commitment to reforms in the field of statistics? What are the main public administration reforms that the Government is working on now and what is the role of statistics in them?

What are the levels of Government and line ministries’ understanding of the importance of effective, accurate and reliable statistics for policy making?

Do the Government reform initiatives, sectoral and cross-sectoral, include measures to improve availability, quality and timeliness of statistics? What are specific examples?

How do the multiple Project partners and beneficiaries view its relevance? Effectiveness Effectiveness focuses on results, not processes and the consultants will examine if the Project has produced its planned outputs and outcomes. The consultants will develop a consolidated Project


72

log frame and will use multiple sources of evidence to examine if the Project is achieving its intended objectives. More specific questions that will be asked include:

To what extent have the planned results been achieved to date (quantitative and qualitative) according to the Project LogFrame/results framework?

What are the factors (positive and negative) that affected output completion?

To what extent and how has the Project contributed to a) improving the statistical infrastructure; b) improving statistical tools; c) data dissemination; and d) data use? What is the evidence?

To what extent has the Project contributed to strengthening capacities of NBS and relevant line ministries/central agencies in the area of statistics and evidence-based policy making?

The Project provides extensive multiple capacity building interventions for diverse beneficiaries. How do the beneficiaries assess the relevance, usefulness and applicability of supports provided?

What were the major factors influencing the achievement or non-achievement of the Project objectives to date?

Are there some Project objectives that were not achieved? What are the reasons?

How do the Project components interact and complement each other?

Did the Project activities manage to achieve systemic changes in the field of statistics and evidence-based policy making? What is the evidence?

Did the central agencies and line ministries improve their capacities to use statistics in policy making? What are specific examples?

What are the innovative approaches developed by the Project? Sustainability Sustainability deals with being strategic and looking to the future and assessing if key Projects elements will be in place and expand beyond the Project completion in 2013. More specific questions that will be asked include:

Did the Project design include an appropriate sustainability strategy (including promoting national/local ownership, use of local capacity, etc.) to support positive changes for both NBS and statistics users after the end of Project?

What is the level of ownership of the reform process within the NBS, line ministries and State Chancellery?

What are the prospects for further development of related interventions after the end of UN support?

To what extent has the Project promoted strengthening of already existing partnerships and establishment of new ones that include NBS? If yes, are these improvements sustainable?

To what extent is the Project sustainable in light of current and Projected demand for improved statistics assuming that Project will be completed in 2013? Does the national Government have a realistic strategic plan of reforms of statistics and policy making?

What is the Project sustainability strategy? Which components of the Project are sustainable? Please provide specific indicators of sustainability.

Are there some components where sustainability remains an issue? What can be done to improve sustainability of these components?


73

What actions (risk mitigation strategies) have been taken to improve Project sustainability?

What are the levels of Government commitment to continue reforms in the field of statistics and evidence-based policy making?

Efficiency Efficiency is associated with the resources used to produce desired outputs. The consultants will examine the extent to which the Project is producing its planned outputs in relation to expenditure of resources. In addition to exploring efficiency in resource use, the consultants will examine what specific steps were undertaken to optimize the Project efficiency. More specific questions that will be asked include:

How efficiently have the financial resources been used?

Did the Project management ensure quality and cost-effectiveness of the process of transforming inputs into outputs and outcomes? What is the evidence?

Have been the timelines of activities always met? If not, why?

Did the Project management ensure proper co-ordination of activities and partners to encourage synergy and avoid overlaps?

How flexible was the Project design in adapting to the changing environment and partners’ needs?

Can the costs of Project deliverables be lowered while still achieving Project objectives?

Are there alternative delivery methods that can achieve the Project objectives more efficiently? What evidence is there to support such methods?

Did the Project apply cost-saving strategies? What are they? Did they work? Impact Impact deals with whether the Project’s overall objectives have been achieved or are likely to be achieved. More specific questions that will be asked include:

What are the key anticipated measurable Project impacts?

What is the probability that the Project will achieve its anticipated impact? What is the evidence?

Are unplanned effects likely to take place?

Has the data collection and data dissemination systems of NBS improved and become aligned with the international standards? What is the evidence?

How was the data analysis system improved? Did new practices and approaches institutionalize in NBS practices?

How did the organization of NBS improve? Did the organizational changes improve the accuracy, timeliness and quality of statistics produced? Did it result in some cost-efficiencies?

Does the national statistical system respect the UN Fundamental Principles of Official Statistics?

What are the social exclusion data available? Is the NBS going to continue producing social exclusion data once the Project is over?

How did the demographic statistics improve as a result of Project interventions?

How did the quality of statistics improve? What is the evidence?

How many statistical indicators are presented in disaggregated format? What is the demand for disaggregated indicators? How are the disaggregated indicators used?


74

What is the Project’s impact on NBS ability to produce reliable and accurate statistics and deliver it in a user-friendly format?

How does the NBS management and staff assess the Project impact, by department? What are specific indicators of the Project success? What are the areas where the impact was limited?

Did the data produced by NBS become more user-friendly as a result of Project interventions?

Did the data dissemination system improve? How? What are the levels of data users’ satisfaction?

Are the users satisfied with the official NBS web-site and the information available there?

How do the practices of data use for decision making improve as a result of Project interventions?

How do the line ministries that the Project collaborated with assess its impact? Did their skills and practices of using statistics in policy/programs development and implementation improve due to the Project interventions? What is the evidence?

Did the Project interventions increase a range of data users? Who are they and how do they use the data?

How did the regional statistics improve as a result of Project interventions? How is it used in policy/program development and implementation?

What are the current capacities of partners that benefitted from the Project’s training opportunities?

Human rights-based approach, equity and gender issues The consultants will examine the extent to which the Project outcomes reflect and contribute to the improvement of availability and use of disaggregated data to capture the situation of the most vulnerable categories of population and factors that contribute to social inclusion. More specific questions that will be asked include:

Was the Project designed according to international norms and agreements on Human Rights (HR) and Gender Equality (GE) and in line with national strategies to advance HR & GE?

To what extent are HR & GE a priority in the overall Project budget and implementation?

What specific disaggregated data pertaining to promoting of HR of socially excluded groups were produced?

What specific gender disaggregated data were produced?

How did the partners use the disaggregated data to advance HR and GE in Moldova? Partnership strategy The Project implementation and its success depend on the effectiveness of UN partnership strategy. More specific questions that will be asked include:

Who are the key partners implementing the Project?

What was the role of each of involved UN agencies?


75

What was the mechanism and process of decision making on Project priorities and activities? Did it prove to be effective? What is the evidence?

Are the national partners satisfied with the Project partnership strategies? What is the evidence?

How did the partnership contribute to the achievement of the Project outcomes? What was the level of stakeholders’ participation?

9.2 List of interviewed individuals and participants of focus groups

N Name Function/Subdivision Organization

1 Aurelia Spataru Project Manager UNDP Moldova

2 Roman Rosca Project assistant UND Moldova

3 Alexandru Oprunenco

Portfolio Manager UNDP Moldova

4 Elena Laur Monitoring and Evaluation Officer UNICEF Moldova

5 Cornel Iftofi Programme Analyst UN Women Moldova

6 Oxana Lipcanu Representative ILO

7 Lucia Sopiala General Director National Bureau of Statistics

8 Oleg Cara Deputy General Director National Bureau of Statistics

9 Vitalie Valcov Deputy General Director National Bureau of Statistics

10 Igor Mocanu Head of Section International Collaboration and EU Integration


11 Ecaterina Galusca Demographic & Population statistics National Bureau of Statistics

12 Nina Cesnocova Demographic & Population statistics National Bureau of Statistics

13 Ala Negruta Social Statistics National Bureau of Statistics

14 Elena Vatcaru Labor statistics National Bureau of Statistics

15 Elena Orlova Environment and Agriculture Statistics National Bureau of Statistics

16 Ludmila Lungu Environment and Agriculture Statistics National Bureau of Statistics

17 Maria Godeac Website Management & Data dissemination


18 Ala Paslariuc Website Management & Data dissemination


19 Lilia Racu Website Management & Data dissemination


20 Mariana Eni ITC Statistics National Bureau of Statistics

21 Lilian Galer Business Statistics and Statistics Methods National Bureau of Statistics

22 Iurie Mocanu Business Statistics and Statistics Methods National Bureau of Statistics

23 Iurii Torkunov Member of Steering Committee, Head of Macroeconomic Projections & Analysis Division

Ministry of Economy

24 Rodica Nicoara Macroeconomic Projections & Analysis Division

Ministry of Economy

25 Lilia Tolociko Macroeconomic Projections & Analysis Division

Ministry of Economy

26 Tudor Cojocaru Head of Policy Elaboration, Evaluation and Monitoring Division



76

27 Angelina Bezu Consultant, Policy Elaboration, Evaluation and Monitoring Division


28 Anna Olefir Human Development Operations Officer Europe and Central Asia

The World Bank, International Bank for Reconstruction and Development

29 Irina Guban Consultant Health and Social Protection The World Bank, Moldova Country Office

30 Victoria Cujba Deputy Head of the decentralization Policies Division

State Chancellery

31 Ion Gumene Head of Policy Coordination Division State Chancellery

32 Ion Beschieru Legal expertise and Assistance Coordinator

Congress of Local Authorities from Moldova

33 Nadejda Darie Women Network Secretary, Expert Consultant

Congress of Local Authorities from Moldova

34 Sergiu Morari Head of Employment and Social Partnership Division

Ministry of Labor Social Protection and Family

35 Victor Turcanu Labour Inspection Ministry of Labor Social Protection and Family

36 Aliona Cretu Consultant, Demographic Section Ministry of Labor Social Protection and Family

37 Diana Doros Deputy chief of Department of Equal Opportunities

Ministry of Labor Social Protection and Family

38 Ludmila Sochirca Director Republican Council on Medical Vitality Expertise

39 Narcisa Mamaliga Deputy Director Republican Council on Medical Vitality Expertise

40 Silvia Rusu Representative National Health Management Center

41 Marcela Tirdea Chief of Analysis, Monitoring and Evaluation Direction

Ministry of Health

42 Valeriu Pantea Head of Science Department Ministry of Health, National Center for Public Health

43 Svetlana Cotelea Head of Public Health Division Ministry of Health

44 Tatiana Zatic Head of Primary Medical Assistance Ministry of Health

45 Rodica Scutelnic Head of Hospital and Urgent Medical Assistance

Ministry of Health

46 Petru Crudu Deputy Director, Health Data Management Department

Ministry of Health

47 Valentina Bodrug Lungu

Gender Expert, CEO NGO Gender Centru

48 Olga Stavila Journalist Radio National

49 Mihai Guzun Journalist Free Lance

50 Igor Guzun Journalist, Director URMA

51 Ludmila Andronic Journalist, PR Manager Free Lance

52 Georgeta Stepanov

Journalist, Lector Journalism Department State University of Moldova


77

53 Victor Ursu Journalist Free Lancer

54 Ludmila Tiganu Communication Specialist UN Moldova

55 Irina Lipcanu Communication Specialist UN Moldova

56 Irina Lazur Communication Specialist UNDP Moldova

57 Alexandru Sandulescu

Advisor, Energy Policy Prime Minister Office

58 Maria Vremis Co-Author of the National Report Free Lance Consultant

59 Anatol Gremalschi Co-Author of the National Report, Program Director

Institute of Public Policy

60 Dumitru Vasilescu Co-Author of the National Report, Project Manager

NHDR

61 Viorica Craievschi Co-Author of the National Report Free Lance Consultant

62 Vasile Cantarji Project Coordinator, Sociologist NGO CBS-AXA

63 Ion Partachi Chief of Department Statistics and Economic Forecasting

Academy of Economics Studies of Moldova

64 Olga Bitca Advocacy Consultant NGO CREDO

65 Vitalie Zeama Consultant NGO Lawyers for Human Rights

66 Galina Saveleiva Representative Institute of Economy and Statistics

67 Dumitru Slonovschi

General Director NGO Magenta Consulting

68 Ion Mirzac Chief of IT Direction National Bureau of Statistics

69 Alexandru Socolov Chief of the Projection and implementation of informational Software


70 Claude Cahn High Level HR Advisor UN Moldova

71 Evghenii Golosciapov

Representative UNDP Moldova

72 Dejan Mincic Representative UNICEF Moldova

73 Nicola Harrington –Buhay

UN Resident Coordinator UN Moldova

74 Speranta Olaru International Aid Cooperation Officer, Project Manager

EU Delegation in RM

9.3 Exploratory questions to inform the Concept Note development

The evaluation team is assigned with a task of developing a Concept Note for a new UN Project in the area of statistics and evidence-based policy making. More specific questions that were asked by the evaluation team include:

What are the current national priorities and needs for statistical sector institutional and functional development?

What are the international independent assessments of the policy processes in Moldova and the role of statistics in them (e.g, OECD SIGMA)? Do they identify specific gaps that can be addressed through a targeted Project focusing on statistics and evidence-based policy making?


78

What are the Government’s vision and priorities in the area of public administration reform in the field of statistics?

What are the perspectives of diverse partners, including but not limited to the Government, Parliament, NBS, central agencies, line-ministries, academics, media, National Bank of Moldova, and others on the needs of improving statistics and evidence-based policy making? Are there specific official documents reflecting these views? Are there political commitments to reforms of public administration that include statistics?

Is the Government ready to implement national and sectoral strategic documents in the fields of macroeconomic, employment, local and regional development, decentralization, research & development and innovation, social protection, education, environment, etc. in terms of their evidence-based monitoring, evaluation and reporting, as well as interventions related to sector level statistics (developing of capacities at central and local levels, piloting new data collection, policy advice support, etc.)?

What are the positions of the EU, WB, UN partners and other development partners on the needs of continuous improvement of statistics and evidence based policy making?

What are specific areas of interest of the current Project development partners such as UNDP, UNICEF, ILO, UN Women, and UNFPA in the future Project?

Are the partners ready and willing to contribute financially and by bringing their expertise to a new Project in the area of statistics? What could be a potential division of responsibilities among partners?

How can the Project scope be expanded and include a broader range of statistics users such as CSOs, media, and researchers?

9.4 Literature, Project materials and other sources used

Materials in English Joint Project Document Strengthening the National Statistical System

Project Registration Fiche

Official NBS website www.statistica.md

Annual Work plans 2008 – 2012 Results and Resources Framework Activity Report on consultancy provided for the National Bureau of Statistics aiming to develop e – Statistics Web Portal Concept (final version) during the period of 25.04.2008 – 20.05.2008 Amarfii, Ion, Activity Report, Development of Software Application for the survey “Violence against Women in the Family in Moldova”, 2010 Call for Expression of Interest on development and institutionalization of statistical literacy university course for journalists and communication specialists, 2012

http://www.statistica.md/


79

Chernyshev, Igor, ILO Department of Statistics, Review of the national labour statistics in relation to the application of the ILO Labour Statistics Convention 160 (1985), 2010 Decentralization and Vulnerability Indicators: Assessment report on data collection, capabilities and potential, Republic of Moldova, 2012 Draft Country Programme Document for the Republic of Moldova for period 2013-2017 Durr, Jean-Michel, international expert, In-depth assessment of Population Statistics of the Republic of Moldova and its adherence to the EU Statistical Requirements Compendium EC Action Fiche for the Republic of Moldova, Pilot Regional Development Programmes (PRDPs), 2013 Evaluation Report, United Nations Development Assistance Framework 2007-2011 – Moldova, 2011 Final report for ‘Train the Trainers’ 23rd – 27th November 2009 Ghimpu, Viorica, consultant UNDP Moldova, Report on the revision of national statistical tools on recording and reporting of occupational accidents in compliance with applicable international standards and European norms Government of the Republic of Moldova, United Nations Development Programme, United Nations Population Fund, United Nations Development Fund for Women, Joint Project Document, Strengthening the National Statistical System, 2007 Government of the Republic of Moldova, Moldova 2020,National Development Strategy of the Republic of Moldova for 2012-2020 Government of the Republic of Moldova, Activity Program Government of the Republic of Moldova European Integration: Freedom, Democracy, Welfare 2011-2014 Government of the Republic of Moldova, United Nations Development Programme, United Nations Development Fund for Women (UNIFEM – part of UN Women), United Nations Population Fund, United Nations Children’s Fund, International Labour Organization, Amendment to the Joint Project Document, Strengthening the National Statistical System of the Republic of Moldova, period of extension 2011-2012 Government of Republic of Moldova and UN Moldova, Improved measurement and monitoring of MDGs in Moldova: targets, indicators, definitions, data sources, progress analysis: Results of technical support missions by specialized statistician experts (MDG 7: Ensure environmental sustainability; MDG 3: Promote gender equality and empower women; MDG 2: Achieve universal access to general compulsory education; MDG 4-5: Reduce child mortality and improve maternal health)


80

Interview within the Daniela Stefanescu, UNDP international consultant, Director of the Department of European Affairs and International Cooperation of the National Institute of Statistics of Romania, UNDP/UNIFEM/UNFPA Joint Project "Strengthening the National Statistical System" Herrmann, Vera, Claudia Junker, Bronislava Kaminskiene, Günter Kopsch, Jason Schachter, Adapted Global Assessment of the National Statistical System of Moldova, 2012 Kwiek, Marek, Strengthening Research and Development (R&D) Statistics in the Republic of Moldova and Final report on undertaken assignment, 2009 Lindqvist, Markku, Finland, In-depth assessment of the income statistical field of the republic of Moldova and its adherence to the EU Statistical Requirements Compendium, 2012 List of indicators of monitoring the decentralization and its impact on population, including vulnerable groups, 2012 Malcoci, Ludmila, Guide for interviewers (in questions and answers), 2008 Malcoci, Ludmila, Curriculum for the training of interviewers, 2008 Measuring social inclusion at sub-national level: Lessons learned from Poland, Based on study tour for Moldovan civil servants, 16-20 June 2008 Memorandum of Understanding between the Participating UN Organizations and the Managing Agent for pooled fund management. Memorandum of Understanding between UNIFEM and UNDP, 2007 Millennium Development Goals and derived statistical indicators, 2010 Minutes of the Steering Committee meetings. Miošić-Lisjak, Nives, International consultant, and Gheorghe Caraseni, Local consultant, Final External Evaluation Report of the Programme Promoting Gender Equality in National Development Policies and Programmes in Moldova, 2007-2010, 2010 Plan of the National Statistical System of Republic of Moldova for the Development of Quality in Statistics, 2009 Plan of Actions of National Statistics Development for the period 2008-2011 Project Reports Public opinion poll concerning the data user’s satisfaction with available statistical data for the National Bureau of Statistics, 2008 Report on revising statistical toolkit in the field of disability statistics in Republic of Moldova


81

Report of activities of national consultant to support the Ministry of Labour, Social Protection and Family in coordination with national stakeholders on the effective implementation of the National Program on Ensuring of Gender Equality SIGMA, The Role of Ministries in the Policy System: Policy Development, Monitoring and Evaluation. SIGMA Paper no. 39, 2007. Singh, Rupinder international consultant and Igor Nedera, national consultant, Institutional Development Outcome Evaluation, UNDP and UNFPA Projects, 2010. Siobhan Campbell, Siobhan Benita, Elizabeth Coates, Phil Davies and Gemma Penn. Analysis for policy: evidence-based policy in practice. Government Social Research Unit. HM Treasury. UK 2007 Social Exclusion in Moldova through official statistics, 2011 Strategy of National Statistics Development for the period 2009-2011 Support to the next phase project formulation “Strengthening the National Statistical System 2011-2012” and NHDR 2010 on social inclusion and NHDR 2011, Mission report, Susanne Milcher, Bratislava Regional Centre Terms of References International consultant to support the National Bureau of Statistics with optimization of its organizational structure, 2010 and reports submitted by the consultant. Terms of References, National consultant to undertake the analysis of primary data on vital events and assessment of their quality, 2012; Terms of References, National Consultant to Revise the Toolkit of Statistical Survey on Accidents at Work in Compliance with Applicable International Standards and European norms, 2012 Terms of References, National consultant to revise the forms of primary accounting on labour time and wages for statistical purposes, 2012 Terms of References, National consultant in accounting to expertise and test the statistical toolkit (questionnaires) to be applied within the integrated statistical survey on enterprises, 2012 Terms of References, National consultant to develop the methodology and tools for the Annual Statistical Survey on Earnings and Labour Cost in compliance with the international standards, 2012 Terms of Reference Elaboration of a feasibility study for establishment and running of a Training Unit to improve statistical literacy and build capacities of data users UNECE Secretariat, Generic Statistical Business Process Model, 2009 United Nations Evaluation Group, Ethical Guidelines for Evaluation, March 2008


82

United Nations Evaluation Group, Integrating Human Rights and Gender Equality in Evaluation – Towards UNEG Guidance, 2011 United Nations Evaluation Group, Quality Checklist for Evaluation Reports, 2010 United Nations Evaluation Group, Standards and Norms for Evaluation in the UN system UN Moldova, The Use of Gender Statistics is Essential for their Daily Job at the Local Level. The UN in Moldova Magazine, April-June 2012 UN Office of the Resident Coordinator in Moldova letter to Mr. Victor Bodiu, State Minister, The Republic of Moldova from 12 January 2012. UNIFEM, Promoting Gender Equality in National Development Policies and Programmes in Moldova; Final Report to the Swedish International Development Cooperation Agency (SIDA), May 2007-April 2010 United Nations – Republic of Moldova, Partnership Framework 2013 – 2017: Towards Unity in Action, Action Plan. United Nations Development Assistance Framework (UNDAF) for Moldova, 2007-2011 UNDP Country Programme Action Plan 2007-2011 UNDP Results-Oriented Annual Report (ROAR) for Moldova (2008, 2009, 2010) UNDP and UNFPA, Report on Institutional Development Outcome Evaluation, 2009 UNDP Assessment of Development Results for the Republic of Moldova, 2011 UN, Evaluation Report United Nations Development Assistance Framework 2007-2011, Moldova, 2011 UNDP, Handbook on Planning Monitoring and Evaluating for Development Results, 2009 UNDP, Outcome-Level Evaluation: A Companion Guide to the Handbook on Planning Monitoring and Evaluating for Development Results for Programme Units and Evaluators, December 2011 UNDP Moldova, National Human Development Reports for Moldova UNDP Handbook on Monitoring and Evaluating for Results UNDP Results-Based Management: Technical Note UNICEF, UNICEF-Adapted UNEG Evaluation Reports Standards, July 2010


83

UNICEF-Adapted UNEG Evaluation Reports Standards, July 2010 United Nations and Government of Moldova, Towards Unity in Action. United Nations - Republic of Moldova Partnership Framework 2013-2017, Workshop on “Monitoring the decentralization reform and its impact on population, in particular the vulnerable groups”, 2012

Materials in Romanian Raportul cu privire la Sondajul de opinie publică privind gradul de satisfacere a utilizatorilor cu date statistice ale BNS” SBS, AXA, 2013 Viorica Ghimpu, Raport detotalizare a lucrarii cu privire la revizuirea instrumentelor statistice naționale de înregistrare și raportare a accidentelor de muncă în conformitate cu standardele internaționale și normele europene aplicabile, 2012 Tatiana Medvediuc, Efectuarea Analizei Datelor Primare Privind Evenimentele Vitale şi Evaluarea Calităţii Acestora, Nota Analitică, 2012.


84

9.5 Terms of References

TERMS OF REFERENCE End of Project Evaluation and Future Programming entry-points in the field of statistics and evidence-based policy making

Sought Applicants: Evaluation team: one international and one national consultants

Application Deadline: 14 February 2013

Expected Duration of Assignment:

March– April 2013, estimated volume of work – up to 30 working days per consultant

Reference to Project: UN Joint Project on Strengthening the National Statistical System,

Funded by: UNDP, UN Women, UNICEF, UNFPA

Project Activity: (6) Project management and evaluation

Contract type: Individual Contracts

Beneficiary: National Bureau of Statistics, UNDP, UN Women, UNICEF, UNFPA, ILO

I. Background for the Evaluation The Joint Project “Strengthening the National Statistical System in Moldova” (hereinafter Statistics Project) was designed on the basis of the needs identified by the National Bureau of Statistics (NBS), in order that this institution can perform its basic role of coordinating the statistical system ensuring standards to statistical production, and providing relevant, accurate, timely, comparable, coherent and complete statistical data. The Project is aimed at strengthening the national statistical system through improvement of data collection, dissemination and utilization of statistical information, with particular focus on national needs and overall conformity of official statistics with international standards. Project includes three components with specific focus on strengthening national capacities in accurate production of qualitative statistical data, their timely dissemination and proper use. The specific expected results from strengthening the national statistical system are:

Improved capacitiesof NBS and other line-ministries, which are responsible for the production of official statistical information, to produce data of appropriate quality in a timely manner and with a necessary level of desegregation by gender, age and geographical dimensions;




85

In the area of pro-poor policy, concrete changes expected from UN cooperation were to improve the quality and usage of statistical data, with an emphasis on demographic, gender, vulnerable groups and geographic disaggregation which had to be achieved in cooperation with the Government through the present Statistics Project as set out in UNDAF 2007-201275, UNDP Country Programme/CP Action Plan 2007-2011,76, UNICEF CPD 2007-2011(2012) and other documents.The Project activities are to assist the Government of the Republic of Moldova in strategic mid-term planning, as well as in the effective monitoring and evaluation of the national strategies’ targets and objectives (national/sectoral development strategies, MDG, etc.), thus responding to the need to simplify and increase the efficiency of the monitoring and reporting process in order to focus on the evaluation of implementation results. At the programming level the Project is to contribute to the achievement of: UNDAF Outcome 1: By 2012, public institutions with the support of Civil Society Organizations (CSOs) are better able to ensure good governance, rule of law and equal access to justice and promotion of human rights; CP Expected Outcome 1.1: Pro-poor policies, addressing development and population issues, are formulated, implemented, and monitored in a more transparent and participatory manner. CP Expected Output 1.1.9: The availability, quality, and usage of disaggregated statistical data is improved (emphasis on geographic, age, and gender disaggregation). Other Project info: Executing Entity: National Bureau of Statistics and other data producers (Ministry of Economy, Ministry of Labour, Social Protection and Family, Ministry of Education, Ministry of Health, Ministry of Internal Affairs, Ministry of Information Technology and Communications, Ministry of Environment, Civil Registration Service etc). Beneficiaries: Central and local public authorities, academia, media, CSOs. Implementation Timeframe: October 2007 –December 2010 and extended for January 2011- June 2013 Budget: USD 1,2 mln.funded by UNDP and UN Women (with support of Swedish Government), plus parallel funding provided by UNFPA, UNICEF, ILO. As part of the UNDP Good Governance Programme and Poverty Reduction Portfolio, the Statistics Project card available on http://www.undp.md/projects/Statistics.shtml. II. Purpose and scope of work The purpose of the evaluation is to provide a holistic, impartial and trustworthy evaluation of the performance, results and achievements of Statistics Project in relation to its objectives. In addition the evaluation is aimed to generate inputs for future programming related to evidence-based policy M&E and strengthening the necessary statistical capacities. Thus, the overall scope of work under this assignment will be focused on the following issues:

A. End of Project Evaluation - underlying factors that impact on outputs and outcomes, capture good and bad practices as well as lessons learned, including unintended consequences, with the objective of assisting UN agencies in improving design and implementation of future UN-supported interventions in this area;

75United Nations Development Assistance Framework 2007-2011, http://www.un.md/key_doc_pub/doc/ENGL.pdf 76Country Programme Action Plan 2007-2011, under UNDP Policy & Analysis Portfolio, http://www.undp.md/publications/doc/CPAP_eng_29-03-2007_GATA.pdf

http://www.undp.md/projects/Statistics.shtml

http://www.un.md/key_doc_pub/doc/ENGL.pdf

http://www.undp.md/publications/doc/CPAP_eng_29-03-2007_GATA.pdf


86

B. Future programming in the field of statistics and improvement of evidence-based policy planning, monitoring and evaluation: advice and provide technical support towards development of a concept note for the expansion of interventions beyond 2012, which UN Moldova can offer as support to the Government in view of strengthening national capacity to build result-based monitoring and evaluation systems, and to collect, analyze and use data, as essential components of the governance structure and provide critical information, empowering policymakers to take better informed decisions.

The Evaluation is expected to shed light on the underlying factors that impact on outputs and outcomes, capture good practices as well as lessons learned, including unintended consequences, with the objective of assisting the Government of Moldova and UN agencies in improving design and implementation of future UN-supported interventions in this area. Accordingly, the UN agencies, the Government of Moldova, key partners and stakeholders in the country intend to use evaluation findings and recommendations as a learning opportunity to improve further cooperation in this domain. III. Objectives

A. End of Project Evaluation The specific objectives of the evaluation are the following:

1. Provide a full and independent assessment of the achievements and results, failures and constraints of the project, as well as an analysis of its performance, management, impact, relevance and sustainability of the interventions;

2. Generate lessons learned and good practices from each of respective outputs established in the project since its inception in 2007 to inform current and future programming at the country level;

3. Assess whether the results achieved are relevant for the current statistics related environment in the country and provide solid basis for UNDP and other partner UN agencies for future programming;

4. Provide sharp and forward-looking recommendations that can guide UNDP, partner UN agencies and other players in developing effective, realistic and evidence-based strategies.

The evaluation should be comprehensive and cover the outcome, outputs, activities and inputs of the project. The results of the evaluation will be used for re-focusing the interventions and guiding future programming. In this context, the evaluation will:

(i) Extract lessons for future interventions in the statistical sector; (ii) Propose improvement of the coordination between donor-supported interventions in

meeting national requirements; (iii) Outline main areas of focus for future UNDP & UN projects.

B. Future programming in the field of statistics and evidence-based policy

development, monitoring and evaluation In terms of future programming, the following is expected from Evaluation Team:

Analyze the national priorities and needs for statistical sector institutional and functional development, formulated by relevant national counterparts (NBS, line-ministries, National Bank of Moldova, Central Election Commission);

Explore/find out the Government’s vision (needed types of interventions) of the public administration reform in the field of statistics in order to create an efficient, functional and


87

sustainable institutional framework for evidence-based policy-making and to ensure high quality, internationally comparable data production and dissemination;

Assess the Government’s readiness and concrete commitments for the implementation of the national and sectoral strategic documents (in the fields of macroeconomic, employment, local and regional development, decentralization, research & development and innovation, social protection, education, environment, etc.) in terms of their evidence-based monitoring, evaluation and reporting, as well as interventions related to sector level statistics (developing of capacities at central and local levels, piloting new data collection, policy advice support, etc.);

Estimate the interest and readiness of development partners to support and engage in activities aimed at enhancing the understanding of different state agencies on reporting and monitoring of MDGs, key national strategies, and poverty and social inclusion that would contribute to improvement of national data sets;

Develop a draft concept note (at least outcome and output level details) on updated UN intervention related to statistical sector/field (official statistics and/or statistics produced by other CPAs) and provide concrete solutions in the context of the new RM-UN Partnership Agreement 2013-2017 with the perspective to be included in the UN Partnership Framework’s Action Plan (under development);

The proposed concept should envisage, but not be limited to, interventions which would:

ensure& enhance the commitment of the national and local authorities and institutions to data driven and evidence-based policy development and monitoring;

ensure the compliance of the national statistics system with international standards, with data collection methods allowing adequate disaggregation (for example by urban/rural, regions/raions, age, sex);

strength the capacities of public administration at central and local levels, media and CSOs to develop and implement new policies and produce progress reports on the basis of available statistics.

Specify the roles and responsibilities of participating stakeholders (UN, government institutions, LPAs, CSOs, international partners, etc.), as well as fund management modalities (considering a joint UN intervention).

IV. Context Overall context The production of reliable statistics and use of statistical data for evidence-based policy design and monitoring is fundamental to human development, the achievement of the MDGs and further approximation to EU standards for the Republic of Moldova. This development challenge has been anchored in international strategic framework documents, such as the United Nations Development Assistance Framework (UNDAF 2007-2011), and national strategic documents (national and sectorial development strategies, etc.). The European Neighborhood Programme Action Plan for Moldova77, also recognizes the need for further assistance in harmonization of official statistics with European standards and to the forthcoming population and housing census.

77EU-Moldova ENP Action Plan was approved in February 2005 for a period of three years and extended by mutual agreement from February 2008 onwards. In December 2009 it was agreed to continue guiding and monitoring the ENP Action Plan on the basis of yearly sets of priorities and schedules.


88

The National Bureau of Statistics (NBS) of the Republic of Moldova plays an important role in the implementation of the national strategic programmes and plans (including national development strategies), being the core provider of the official statistical data in the country and outside its boundaries, in particular for the EU statistical system. According to the global assessment of the official statistics system in the Republic of Moldova conducted in 2007 the NBS was doing its best to follow international standards, priority being given to those EU requirements that also coincided with national priority needs. In The National Statistical System was evaluated as being reasonably developed, adhered to the Fundamental Principles of Official Statistics and in many domains fully or nearly fully complying with EU requirements78. Despite of this significant progress in the improvement of the statistical system in the previous years, the data collection and data dissemination systems of NBS remained weak and not in line with international standards. The Moldova’s data system still suffered from inconsistent methodology, multiplicity of data sources, weak capacity of data providers and data users to acknowledge the value/meaning and properly benefit of available data. A lot of statistical indicators were not disaggregated by required dimensions or, if disaggregated, then often not disseminated and not easily accessible or understood by users. The low statistical users’ literacy generated poor use of available data for analysis of the current situation, participatory policy making, monitoring of policies’ implementation and evaluation of their impact. Also, the National Statistical System faced many issues related to underdevelopment of computerized registers, fragmented statistical activities in line ministries, inadequate resources and lack of continuous training and capacity building. Project context As a response to all these circumstances the five UN agencies, namely UNDP, UN Women, UNFPA, UNICEF and ILO, have joined efforts to provide more strategic and coordinated interventions to the Government of Moldova through the Project on Strengthening the National Statistical System of the Republic of Moldova aimed to strengthen data production, dissemination and use. Since the mentioned above assessment, the NBS has been supported to undertake numerous measures to improve the situation in different statistical fields. Thus, the NBS’s obtained concrete achievements and notable progress as regards: (i) the review of applied methodologies and their harmonisation with the international and EU standards, (ii) the development of new statistical tools and surveys to collect new data or update out of date sets, (iii) the introduction of innovative tools of data dissemination, (iv) the initiation of thematic analysis of generated data, (v) the enhancement of the dialogue and cooperation with other data producers and with data users79, (vi) the functional optimization of the institution in order to attain the strategic objectives in accordance with in the

78Global assessment of the system of official statistics in the Republic of Moldova, May 2007, by Willem de Vries, http://www.statistica.md/public/files/SeminareConferinte/conf3_4mai2007/Raport_Willem_de_Vries_en.pdf. General assessment results: 35% of the statistical domains fully correspond to the EU requirements, half of them - correspond to a great degree and 15% - partially correspond to the EU standards. 79The NBS has launched the user satisfaction survey, collecting the various points of view with regard to the visibility of statistics and the way in which it is perceived, the quality of official statistics, its accessibility on the Internet, statistical publications, the monitoring of users’ requests, etc., http://www.statistica.md/public/files/despre/evaluare_opinii/Raport_OIKOS_eng.pdf

http://www.statistica.md/public/files/SeminareConferinte/conf3_4mai2007/Raport_Willem_de_Vries_en.pdf

http://www.statistica.md/public/files/despre/evaluare_opinii/Raport_OIKOS_eng.pdf


89

National Development Strategy 2008-2011, the Strategy for the Development of Statistics 2008-2011 and other national and sector policy documents, etc. Undertaken efforts resulted in the increased availability of multi-dimensional disaggregated data (by sex, age, socio-economic status, geographic environment, etc.), production and dissemination of new data on topics unexplored before (in particular social exclusion, child labour, violence against women in family, time use, competencies and skills of migrants, short-term business statistics), gradual implementation of quality management system in national statistics through statistical surveys compliant with UN/international standards & recommendations. Also, all these interventions have helped to mainstream gender and vulnerability issues, and thus human-rights approach, in statistics and to country development as a whole80. The analysis and monitoring framework used for the reporting of the progress made by Moldova in meeting the Millennium Development Goals has been strengthened through the (methodological) review of national indicators for five MDGs (2, 3, 4, 5, 7) and advice on improvements needed in targets evidence-based monitoring and analysis. These is supposed to make the promotion of relevant and efficient MDG-oriented policies in different fields easier to accomplish and more possible to follow the progress towards the achievement of MDGs. With the government priority of European integration good progress in the implementation of the plans agreed with the EU has been made, in particular on alignment of the national statistics to EU standards and acquis communautaire81. Continuous need to harmonize the statistical infrastructure and methodologies of the National Bureau of Statistics, and other data producers, with international and EU standards remains one of the priority areas of the Strategic Development Programme of the NBS for 2011-2014. The Joint Project on Statistics was instrumental in this field, making a critical contribution to improving the quality and accessibility of statistical data in Moldova which was possible due to combined complementary areas of expertise of the UNDP, UN Women, UNFPA, UNICEF and ILO. Context of the future programming in the statistical sector In the process of new UN-RM Partnership Framework for 2013-2017, recently endorsed, the UN has reaffirmed its interest to further support the Government in its implementation considering that qualitative (official and administrative) statistical data are essential means/evidence to support analytical social policy and planning of the government, to promote human development and rights,

80Support has been provided for process of implementation of the National Programme on Ensuring Gender Equality for the period of 2010-2015, empowered by a mid-term Action Plan (2010-2012), by the elaboration of 8 sector sets of core and secondary statistical indicators (gender sensitive) as a practical tool (accompanied by a corresponding user guidelines on its application) to be used by the concerned ministries for monitoring of progress achieved in gender equality. 81According to the negotiations of the EU-Republic of Moldova Association Agreement (under Title 3: Economic cooperation – Statistics), the national statistical systems should respect the UN Fundamental Principles of Official Statistics, taking into account the acquis in statistics, including the European Statistics Code of Practice, in order to align the national statistical system with the European norms and standards. As the experience of other young EU Member States has shown, major investments are required for implementing the Acquisin statistics in a country that is in the process of acceding to the EU.


90

equality, social inclusion and cohesion, to nurture a democratic culture and improve the live of people. During the national consultation process to develop the new UN-RM Partnership framework 2013-201782, the need to address the issue of strengthening national capacity to build result-based monitoring and evaluation systems, and to collect, analyze and use data, as essential components of the governance structure and provide critical information, empowering policymakers to take better informed decisions, has been underlined. Thus, under pillar 1 regarding the Democratic Governance, Justice, Equality and Human Rights, UN agencies will seek to improve the capacity of the government in policy development and in the monitoring and evaluation of the impact of policies. Moreover, statistics (like gender) is a cross-cutting issue throughout the whole RM-UN Partnership Framework, statistical data disaggregated by diverse vulnerability dimensions being required for the indicator /baseline /target levels, as well as means/tools of verifications of measures to be undertaken in a medium-term and resulted impact, in particular related to outcomes on justice, employment, research & development, regional development, education, social protection, environment, etc., which implies consistent support to meet the development needs of respective statistical fields. Therefore, the statistical sector would need to be strengthened further, similar to other sectors, and supported to provide those data/evidence required to monitor UN/Government’s targets, objectives, etc. because the vulnerability issues, regional statistics, decentralization, EU integration (on which the new partnership framework is focused) represent new issues for the national statistics as well as for the whole country. On informative basis, the improvement of the statistical system is part of the Government’s priority lists taken a basis for formulation of UN-RM Partnership framework 2013-2017. The Government’s Activity Program 2011-201483, under the chapter on Reform of the central public administration, among the priority actions, mentions the improvement of “the tools for collecting and sharing data about the main monitoring and evaluation indicators of national policies, including by making the statistics and databases public”, as well as “Build policy analysis, monitoring and evaluation skills within line ministries”. Given the above circumstances, within this assignment, the needs of the national counterpartson further strengthening of the national statistics and the evidence-based monitoring & evaluation capacities of the Government will be subject to exploring & a thorough review to identify and formulate relevant ‘entry-points’ for further interventions and support by UN organisations, reflected in a concept note or draft project document to be used also for programming purpose and funds’ mobilization. V. Evaluation scope

82Towards Unity in Action: United Nations-Republic of Moldova Partnership Framework 2013 – 2017, http://www.un.md/key_doc_pub/ 83Activity Program Government of the Republic of Moldova European Integration: Freedom, Democracy, Welfare 2011-2014, http://www.gov.md/lib.php?l=en&idc=445

http://www.un.md/key_doc_pub/

http://www.gov.md/lib.php?l=en&idc=445


91

The evaluation is expected to address the following issues: Outcome status: What and how much progress has been made towards the achievement of the outcome (including contributing factors and constraints) to which the project was contributing84:

Determine whether or not the outcome has been achieved and, if not, whether there has been progress made towards its achievement;

List innovative approaches tried and capacities developed through joint UN assistance in the field of statistics;

List the factors (positive and negative) that affect output completion. Underlying factors - analyze the underlying factors beyond UN’s control that influenced the outcome. Distinguish the substantive design issues from the key implementation and/or management capacities and issues including the timeliness of outputs, the degree of stakeholders and partners’ involvement in the completion of outputs, the changing political environment and how processes were managed/ carried out. Although the evaluators would not have the task to make judgments about the quality of the data produced with Project’s support, some conclusions on accuracy and reliability of data might be drawn on the basis of specialist-experts’ reports/visions/assessments and/or data user satisfaction — this would help to confirm the robustness of project outcomes and their longer term sustainability.Outputs status: Were the outputs relevant to the outcome? Were the outputs achieved? What are the factors (positive and negative) that affect the accomplishment of the outputs? Activities status:Were the activities to achieve the outputs effective and efficient? How well the activities were planned and implemented? Were key methodologies and approaches that facilitate the success of the initiative, particularly regarding participation and empowerment, gender balance, and delivery of necessary inputs appropriate? Inputs status: What contribution UN partner-agencies have made to the progress towards the implementation of activities and achievement of the output and outcome? How appropriate were the inputs? Were the inputs sufficient to achieve the results? How cost effective they were? Partnership strategy: Ascertain whether UN partnership strategy has been appropriate and effective. What partnerships have been formed? What was the role of each of involved partner UN agencies? How did the partnership contribute to the achievement of the outcome? What was the level of stakeholders’ participation? Examine the partnership among UN Agencies and other donor organizations in the relevant field. Objectively verifiable indicators: Where all of the indicators/targets set at the beginning of the project met? If not, why? Where any exceeded? Statistical Issues: assess whether the applications and solutions were appropriate for the local context and local capacities, how were they selected and deployed and how sustainable are in the medium and long term in terms of cost, maintenance and user capacity.

84See www.undp.md for the Country Programme Action Plan and Project Results Framework.

http://www.undp.md/


92

Capacity development: as a key todevelopment effectiveness, the achievement of the Millennium Development Goals and project sustainability. VI. Evaluation criteria The evaluation criteria, specific to the given situation and being tailored by the scope of activity, against which the subject to be evaluated will be assessed, include:

Relevance - degree to which the overall purpose of the project remains valid and pertinent in the current context and can provide basis for further statistics development in the country;

Effectiveness - extent to which the project results have contributed to the national context and the outcomes as defined by the Project document;

Sustainability - extent to which the changes, benefits and results brought by the project can be expected to last/endure after project completion and when the involvement of UN partner agencies has ended.

The evaluation team will also provide recommendations for potential follow-up interventions in a larger context of other development partners current and planned ’interventions, i.e. how feasible the follow-up actions would be, what alternatives can be identified and/or what components can be added to it, what knowledge products could be developed;

Efficiency - measure of productivity of implementation process: extent to which the project management ensured quality and cost-effectiveness of the process of transforming inputs into outputs and outcomes, timelines of activities, use of financial resources, proper co-ordination to encourage synergy and avoid overlaps; project’s flexibility in adapting to changing needs;

Impact -extend and ways in which the project succeed to improve the national statistical system in terms of data production and data use for pro-poor policy formulation and monitoring.

In addition to this, the following approaches will be considered:

human rights-based approach and gender issues - extent to which the project outcomes reflect and contribute to the improvement of availability and use of disaggregated data to capture the situation of the most vulnerable categories of population and phenomena which would affect their living such as social exclusion, marginalization, etc.;

potential application of lessons learned -main lessons learned which should be avoided/considered and good practice examples which could be replicated and relevant for further planning and programming in the field of statistics and policy monitoring.

The rationale for not using a particular criterion, if such, should be explained in the evaluation report, as should any limitations in applying the evaluation criteria. Performance standards or benchmarks used in the evaluation should also be described. The comprehensive and tailored set of evaluation questions within the framework of stipulated evaluation criteria will be developed by the consultants and consulted with the UN partner agencies. VII. Methodology of work An overall guidance on evaluation methodology can be found in the Norms and Standards of the United Nations Evaluation Group (UNEG)85, UNDP Handbook on Monitoring and Evaluating for Results,UNDP Guidelines for Outcome Evaluators and other relevant document. The evaluators should come up with a suitable methodology for the evaluation of this intervention based on the guidance given in the mentioned reference documents and agreed with partner agencies.

85UNEG Norms and Standards, http://www.uneval.org/normsandstandards/index.jsp?doc_cat_source_id=4:


93

To fully undertake the assignment, the consultants are expected to apply the following methodologicalapproaches for data gathering and analysis:

Desk review of relevant documents (project documents with amendments made, narrative and review reports -midterm/final, donor-specific, etc.);

Semi-structured and/or in-depth interviews with:

Senior management, programme and Project staff of UN partner agencies;

partners and stakeholders (including gathering the information on what the partners have achieved with regard to the outcome and what strategies they have used);

project teams, project beneficiaries. In addition, briefingand debriefing sessions, consultation and validation meetings will be conducted by Evaluation team with NBS, the Government (in particular line-ministries), relevant UN partner agencies, CSOs, academia, media, other donors and partners. VIII. Expected deliverables The key products expected from this assignment are as follows: Evaluation report (about 30 pages plus annexes) that should include, but not be limited to, the following components:

Executive summary (brief description of the purpose, context, objectives of the evaluation, main findings, conclusions, recommendations and lessons learned);

Introduction;

Description of the evaluation methodology;

Analysis of the situation with regard to the outcome, the outputs and the partnership strategy;

Analysis of opportunities to provide guidance for the future programming;

Key findings, including best practices and lessons learned (in particular addressing issues related to effectiveness, efficiency and relevance)

o assessment of performance:

Project Formulation and Implementation (implementation approach, country ownership, stakeholder participation, replication potential, linkages between project and other sector interventions, cost effectiveness, M&E, coordination and operation issues, risk management);

Results (attainment of objective, appropriateness of outcomes, prospects of sustainability);

o Estimation of intervention’s impact;

Conclusions and recommendations (the last based on evidence and analysis) o Evaluation team will highlight key strategic recommendations, suggesting an

appropriate sequencing in the implementation of recommendations whenever possible;

o It would be desirable that the report will also draw separate conclusions for each UN agency involved;

o To ensure programmatic and technical relevance, the developed recommendations will be consulted with the key stakeholders;

Annexes: ToRs, brief notes on interviewed people, reviewed documents, etc.


94

More guidance on the content of evaluation report, see in the UN evaluation reports standards in the list of references86. The evaluators will prepare a presentation of the (preliminary/final) findings to be discussed at a roundtable with Project partners. The evaluators are required to discuss the full draft of the evaluation report with main stakeholders prior to departure from Moldova. Concept note (15-20 pages plus annexes) for the future UN interventions in support of national statistical system based on the recommendations of the mission is to be produced. The document will describe the assistance that UN Moldova can offer as support to the Government in view of strengthening national capacity to build result-based monitoring and evaluation systems, to collect, analyze and use data, as essential components of the governance structure and provide critical information, empowering policymakers to take better informed decisions. Opportunities to engage other UN agencies, different than those already in the Statistics Project, in future work will be considered.The format of the concept note will be agreed between UN partner agencies and the evaluators prior to the start of the evaluation. Proposed Outline of the concept note includes but is not limited to:

- Executive Summary & Introduction; - Situation Analysis (overall background and specific context); - Scope and Strategy (describe clearly the rational for the proposed intervention, suggest

the outputs with related indicators, baselines and targets; define the methodological approach for implementation, including how outputs of a new project will be achieved);

- Roles and Responsibilities of key partners & Management Modality; - Key activities & budgetary indications.

Although the Project’s evaluation is required to investigate whether data has been disaggregated by sex and other dimensions, still, the commitments (of collectors, interpreters and users of statistical information) to the importance of gender, as well as of ethical issues will be considered by the experts in the concept note for future programming. Both products shall be submitted in English in electronic forms. IX. Composition, qualifications of the evaluation team and tasks distribution Members of the evaluation team must not have been associated with the project’s formulation, implementation or monitoring. To achieve the above objectives, the team will be composed of a senior team leader (international), assisted by a national consultant. If need is confirmed, the Project might benefit of the advice and consultation of the Human Development advisor from the UNDP Bratislava Regional Center regarding the future programming perspectives. Required qualifications and skills:

for the International Consultant(team leader):

Advanced university degree in economics, statistics, social sciences, public administration, international development or related fields;

At least 5 years of working experience in the review, evaluation and assessment of the development projects and/or programmes, with budget over USD 0.5 million;

At least 7 years of work experience in the field of democratic governance, public administration, development, including participatory planning, monitoring and evaluation;

86Standards for Evaluation in the UN system, Chapter 4, UNEG Quality Checklist for Evaluation Reports, www.uneval.org


95

Sound knowledge about results-based management (especially results-oriented monitoring and evaluation) confirmed by evidences (certificate/s) - would be an asset;

Proved practical experience of work related to the use of statistical data, evidences and informative materials, but also related to data collection and/or production;

Experience of collaboration with Government institutions (in particular National Statistical Offices) in areas relevant for the present assignment;

Familiarity with the political, economic and social situation in the Republic of Moldova – would be an asset;

Previous work experience in related areas with UN partner agencies or other development organizations preferably in the region;

Good interpersonal skills, solid judgment/ decision making, having demonstrated initiative; creative and team player; self-starter and results-oriented;

Ability to analyse, plan, communicate effectively orally and in writing, produce well written analytical reports, solve problems, organize and meet expected results, manage diversity of views, adapt to different environments (cultural, economic, political and social);

Integrity by modelling the UN's values and ethical standards;

Fluency in English; knowledge of Romanian or Russian would be an advantage.

for the National Consultant:

Advanced university degree in economics, statistics, social sciences, public administration, international development or related fields; At least 3 years of working experience in the review, evaluation and assessment of the development projects and/or programmes;

At least 5 years work experience in the areas related to good and democratic governance and its development, public administration in R.Moldova

Knowledge about monitoring and results-based management (confirmed by evidences) - would be an asset;

Experience of collaboration with Moldova’s Government institutions in areas relevant for the present assignment;

Proved practical experience of work related to the use of statistical data, evidences and informative materials, but also related to data collection and/or production;

Previous work experience in related areas with UN partner agencies or other development organizations preferably in Moldova or outside the country;

Familiarity with the national context (legal and policy framework) related to the areas of assignment concern;

Good interpersonal skills, solid judgment/ decision making, having demonstrated initiative; creativity; self-starter and results-oriented;

Ability to analyse, plan, communicate effectively orally and in writing, produce well written analytical reports, solve problems, organize and meet expected results, manage diversity of views, adapt to different environments (cultural, economic, political and social);

Integrity by modelling the UN's values and ethical standards;

Fluency in English, Romanian and/or Russian languages.

Distribution of tasks:

The International Consultant(team leader) will perform the following tasks:

Lead and manage the evaluation mission;


96

Design the detailed evaluation scope and methodology (including the methods for data collection and analysis) for the report;

Decide the division of labor within the evaluation team;

Conduct an analysis of the outcome, outputs and partnership strategy (as per the scope of the evaluation described above) for the report;

Draft the Project evaluation report and concept note;

Finalize the whole evaluation report and concept note. The team leader will take the overall responsibility for the quality and timely submission of the evaluation reports to the UN partner agencies.

-the national consultant will perform the following tasks:

Review the available documents;

Participate in the design of the evaluation methodology;

Collect relevant data and information;

Actively participate in conducting the analysis of the outcome, outputs and targets (as per the scope of the evaluation described above), as agreed with the international consultant;

Draft related parts of the Project evaluation report and contribute to the drafting of concept note ,

Organize planned meetings and interviews;

Assist the team leader in finalizing document through incorporating suggestions received on draft evaluation report and concept note related to his/her assigned sections.

X. Proposed Time frame It is expected that the assignment will be conducted during the period February – April 2013, over a period of up to 30 working days (per consultant) (full days of 8 hours) during which all the activities and outputs/results envisaged under the present assignment are expected to be performed. For International consultant this will also include in-country mission of up to 7 days (mission costs will be stipulated separately in the financial proposal and include DSA according to UN rates, international transport, other costs if/when justified). The final evaluation report should be delivered by the end of April, 2013. The detailed schedule of the evaluation and the length of the assignment will be discussed with the evaluators prior to the assignment. The indicative activity timeframe and responsible party is:

Activity Modality of work

Estimated number of working days Tentative Timeframe from contract date

International Consultant

National Consultant

Evaluation design, methodology and detailed work plan (agreed with UN partner-agencies and NBS)

On-line 1 day 1 day In 2 days

Desk review of documents On-line 4 days 4 days In 10 days

Interviews with partners and key stakeholders, consultations

in-country mission

5 days 5 days In 15 days

Draft the Project evaluation report On-line overall 10 days for the paper

overall 7 days for the paper

In 20 days

Draft the Project concept note for On-line overall 5 days for overall 5 days for the In 25 days


97

future programming the paper paper

Incorporate feedback and inputs from Project team (UN COs, NBS) in the draft evaluation report and concept note

Online overall 2 days (or 1 day for each paper)

overall 2 days (or 1 day for each paper)

In 30 days Debriefing with partners On-line or in-

country mission

1 day 1 day

Finalization of the evaluation report (incorporating the received comments)

On-line or in-country mission

1 day for the paper 1 day for the paper Before 30.04.13

Finalization of the concept note (incorporating the received comments)

On-line or in-country mission

1 day for the paper 1 day for the paper Before 30.04.13

Total 30 days 27 days

Note: The mentioned number of working days has been estimated as being sufficient/ feasible for the envisaged volume of work to be completed successfully and is proposed as a guideline for the duration of assignment, and it can not be used as criteria for completion of work/assignment. The provision of envisaged deliverables approved by the Project partners and concerned national stakeholders would be the only criteria for the Contractors work being considered completed and eligible for payment/s. XI. Implementation arrangements, management and logistics responsibilities The team of consultants is expected to work in a close cooperation with the national counterparts and the UN partner agencies staff and to keep them updated on the progress at appropriate intervals during the assignment. The consultants will work both inside (being provided with working space, access to Internet, printer and telephone line) and outside the project office, based within the premises of the NBS. TheUNDP project staff, as the overall manager of the project and the first point of contact,and COs teams will provide support to the mission by: i) liaising with national stakeholders in the country; ii) assisting the team with the identification and collection of necessary information, data and documentation related to the project, as well as with the conduct of roundtable discussions / consultations / field visits / interviews; and iii) any logistical and administrative support that may require by the team. This is a joint procurement that will be undertaken by UNDP on behalf of UN Women, UNICEF and UNFPA. However, the evaluation will be fully independent and the evaluation team will retain enough flexibility to determine the best approach in collecting and analyzing data for the Project evaluation.

The Project partners and its counterpart within the Government of RM -- NBS as the partner institution and key beneficiary -- will be supporting the selected evaluators by: On behalf of each partner, nominate the responsible staff to support the evaluator/s in accomplishment of the present assignment; This staff will be included in the Coordination Team (created for the purpose of coordinating this assignment),


98

Provide necessary informational and logistical support; Provide the relevant background documents available related to the purpose of envisaged

assignment, results of previous work, information and materials, etc. Such papers are to be used solely for this activity and are not for other purposes unless permission is granted by NBS and/or UN partner agencies;

Provide the necessary guidance at the request of the evaluators related to the objectives and purpose of the work;

Promptly endorse the materials submitted by the contracted evaluator/s.

end of project evaluation and future programming …...un joint project on strengthening the...

Documents