Final External Evaluation of the Justice Partnership Programme

Debriefing Workshop Presentation

Prepared by Finn Hansen & Pierre Robert | 5 June 2015

JPP Evaluation Debriefing Seminar – Hanoi – June 2015 2

“The contents of this publication are the sole responsibility of the authors and can in no way be taken

to reflect the views of the European Union.”

JPP Evaluation Debriefing Seminar – Hanoi – June 2015 3

Seminar Objectives

Why this seminar?

• To present the preliminary findings, conclusions and recommendations of the JPP evaluation team.

• To discuss the findings and conclusions.

• To assess whether and how the JPP could have performed better.

• To discuss lessons learned for future cooperation

JPP Evaluation Debriefing Seminar – Hanoi – June 2015 4

Proposed Agenda

• 8:40: Presentation of the evaluation

• 10:00: Break

• 10:20: Discussion

• 11:20: Conclusions and next steps

• 11:30: Close

• Lunch

JPP Evaluation Debriefing Seminar – Hanoi – June 2015 5

The Programme

• JPP: 2010-2015, funding from Denmark, Sweden and EU – €18.7m (programme document figure)

Objectives:• Justice sector state institutions strengthened in their efforts to implement

judicial reforms

• Empowerment of a self-managing Vietnam Bar Federation (VBF) and lawyers supported

• Capacity of Non-Government Organisations (NGOs) to contribute to awareness of rights, access to justice and judicial reform enhanced.

JPP Evaluation Debriefing Seminar – Hanoi – June 2015 6

3 Programme Components

• Component 1: support to Ministry of Justice (MoJ), Supreme People’s Court (SPC) and Supreme People’s Procuracy (SPP) to improve procedures, strengthen institutions and develop staff capacity.

• Component 2: support to the VBF through the Canadian Bar Association to enhance organisational capacity and provide training to lawyers.

• Component 3: provide support to NGOs working on justice issues, essentially through the Justice Initiative Facility Fund (JIFF), including advice and training to grantee organisations.

JPP Evaluation Debriefing Seminar – Hanoi – June 2015 7

The evaluation

• Objective: to assess the JPP’s performance with regards to its key results areas (KRAs). To draw lessons for use in the implementation of the forthcoming EU JULE programme, due to start later in 2015.

• Methodology:– Study of documentation– 2 visits by evaluation team to Vietnam: April and May/June 2015– Debriefing– Evaluation draft and final report

JPP Evaluation Debriefing Seminar – Hanoi – June 2015 8

Evaluation criteria• Relevance:

– Correctly identified needs?– Appropriate response to those needs?

• Effectiveness:– Planned activities implemented?– Achievement of expected results?

• Efficiency:– Good value for money?– Management quality?

• Sustainability:– Established new working methods, spread new knowledge?– Will results continue after the programme ends?

• Impact:– Did the programme change attitudes?– Benefits for citizens?

JPP Evaluation Debriefing Seminar – Hanoi – June 2015 9

FINDINGS

JPP Evaluation Debriefing Seminar – Hanoi – June 2015 10

Component 3

• Supported NGO action on rights awareness, legal advice, research, dialogue.

• JIFF secretariat, grant committee established

• 78 grants in 47 provinces (10 grants for national activities)

• Innovative approach, new in the justice sector

• Examples:– Centre for Consultancy on Law and Policy in Heath on HIV/AIDS– Law Development Journal– ACDC: Legal advice for people with disabilities– Justice for « cancer villages »– CSAGA: mediation againt domestic violence– Hoa Binh Association and Legal Aid Centre: community consultation group

JPP Evaluation Debriefing Seminar – Hanoi – June 2015 11

Component 3

• Relevance:– Enhanced NGO contribution and participation in justice issues– KRAs appropriately selected– Contribution to dialogue on the performance of the justice system

• Effectiveness:– Increase in number of people involved in awareness raising activities– Increase in provision of legal advice and number of providers– More access to legal advice in remote areas– Advice to women, minority communities– Research and publications on judicial reform (including gender aspects)– Dialogue at central and local levels on judicial reform

JPP Evaluation Debriefing Seminar – Hanoi – June 2015 12

Component 3• Efficiency

– Good use of funds, efforts to keep project costs down– Good ratio of outputs to costs– Some synergies with other donors– 30% of budget (about €3m total) for JIFF management, large but needed

• Sustainability– Project level: short-term grants, little capacity to continue– Project management skills improved, credibility of grantees enhanced

• Impact– Some changes in attitude (e.g. less prejudices against HIV+, more

confidence in the law)– Some acceptance of NGOs as legitimate interlocutors for local dialogue

JPP Evaluation Debriefing Seminar – Hanoi – June 2015 13

Component 2

• VBF established 2009

• Additional to local-level Bar Associations

• Mandated by 2006 Law on Lawyers

• CBA conducted organisational assessment, training needs assessment

• Assessment of administrative systems, establishment of management processes

• Training for Board members, engagement with local BAs

JPP Evaluation Debriefing Seminar – Hanoi – June 2015 14

Component 2

• Relevance– Very clear need for organisational development– CBA was appropriate support: credible, good understanding of VBF– Weakness: overambitious goals in the time given

• Effectiveness– Development of operating rules: governance, administrative systems, staff

training– Support to regulatory framework: Code of Conduct and Ethics,

dissemination and training for lawyers (but: weak internal communications)– Support to VBF to speak on behalf of lawyers: contribution to legislation

(issuance of lawyer‘s card; Chapter VII of Criminal Procedure Code) – Professional training: support to professional development of lawyers,

development of training materials (but weak training capacity)

JPP Evaluation Debriefing Seminar – Hanoi – June 2015 15

Component 2

• Efficiency– Good management support by CBA– Improved management capacity of VBF staff– Need for further assistance, broader involvement of lawyers in VBF

management– Weak communications mechanisms, dificulties with membership fee

collection

• Sustainability– Important step towards institutionalisation, recognition of the profession at

national level as integral to the judicial sector.– No achievement of financial sustainability yet, need to diversify/reinforce

funding.

• Impact– Gradual change in perception of the role of VBF– Advocacy role on behalf of lawyers is still at early stage

JPP Evaluation Debriefing Seminar – Hanoi – June 2015 16

Component 1

• Worked with MoJ on procedures, institutions and capacity:– Legal aid, civil procedure, Law on Lawyers– Socialised bailiff system pilot– Judicial records– Capacity building (legal aid, lawyers training, support staff)

• Worked with SPC on legislation and management:– Civil Procedure Code– Administrative Procedure– Case file management (court dossier)– Publication of judgements– Training, seminars on international practices

• Worked with SPP on legislation and management:– Criminal Procedure Code– Organisation of the Procuracy– Capacity development, training, – Seminars on international practices

JPP Evaluation Debriefing Seminar – Hanoi – June 2015 17

Component 1

• Relevance– The KRAs are very relevant– But formulation of purpose was too broad (however, may be necessary for

a demand-led programme)– Good level of implication of institutions in programme design– Aligned with the priorities of the Judicial Reform Strategy

• Effectiveness– In all, 12 of 19 KRAs have been achieved, 6 are under completion, 1 is

pending (Criminal Procedure Code).– MoJ: Legal Aid strategy completed (to be approved), Law on Lawyers

implementation underway; capacity of provincial legal aid centres improved (from a low base)

– SPC: considerable JPP input in at least 3 KRAs (e.g. case file management (court dossier), publication of judgements); significant input in Criminal Procedure Code revision (also input from VBF supported by JPP); capacity development plan achieved.

JPP Evaluation Debriefing Seminar – Hanoi – June 2015 18

Component 1

• Effectiveness, continued– SPP: revision of CPC, including support for alignment of key provisions

with international standards (right to silence, to defence counsel, etc); Law on the organisation of the procuracy; capacity building of SPP‘s controlling function (e.g. control of places of detention).

– Key concern: training not always adapted to Vietnam‘s situation.

• Efficiency– Overall, critical mass reached in the of TA and international expertise

provided.– Committed, highly skilled and experienced TA team added value to

programme– But: delays in approvals of some activities, issue related to use of national

civil servants as experts.

JPP Evaluation Debriefing Seminar – Hanoi – June 2015 19

Component 1

• Sustainability– New legislation by definition continues beyond programme period.– Significant improvement to skills (senior staff refer to improvement in

quality of work in their department, e.g. on legal drafting).– New practices initiated/developed (e.g. publication of judgements, use of

precedents) with substantial technical input from JPP.– But: issue of financing for the future (e.g. legal aid, bailiff system)

• Impact– Enhanced awareness of international best practices in a range of fields of

relevance to Vietnam– Improved practices in legal drafting/consultation on new (and future?)

drafts.

JPP Evaluation Debriefing Seminar – Hanoi – June 2015 20

Conclusions

JPP Evaluation Debriefing Seminar – Hanoi – June 2015 21

Programme-level

• The programme was relevant:– Good understanding of the needs– Good design of the response of the 3 components

• Component 1 demand-led• Component 2 helped organisational development• Component 3 sound design, pro-active support to NGOs

• Programme was effective:– Good level of achievement of activities– Most anticipated results were met

• Programme was efficient:– Appropriate use of funds– Good programme management

• Satisfactory levels of sustainability and impact

JPP Evaluation Debriefing Seminar – Hanoi – June 2015 22

Key lessons learned

• Overly ambitious or unclear in objectives and some outcomes

• Original design failed to provide for linkages among the components

• Lack of appropriate programme-level monitoring and evaluation mechanism

• Overly complex programme management structure (PC, 5 PMUs, 3 TA teams)

• Comp. 3: some grants were too short term• Comp. 2: ended too early for VBF to be solidly established• Comp. 2: need for a VBF organisational development plan• Comp. 1: insufficient linkages fostered among 3 institutions• Comp. 1: training somewhat ad hoc, lacked predictability over 12-18 months

JPP Evaluation Debriefing Seminar – Hanoi – June 2015 23

Recommendations

• To EU, donors:– Continue to support the justice sector development. MoJ, SPP, SPC, VBF– Continue to support NGOs working on justice issues

• Project vs. “core“ funding• Thematic priorities?

– Ensure future programme management has stronger Monitoring and Evaluation (M&E) and synergies across components• The beneficiary institutions should be involved in the selection of indicators, in

agreement with donors

– Address need for more predictable, more widespread training (may require different planning processes within Component 1 institutions)

JPP Evaluation Debriefing Seminar – Hanoi – June 2015 24

Recommendations

• To the MoJ, SPC and SPP: – The three institutions should reduce the number of areas of work in which

they seek assistance from future cooperation, instead prioritising on-going support in a narrower range of fields.

– The three institutions should attempt to develop 12-months rolling training plans, to enhance the predictability of training sessions. They should consider whether to repeat some training sessions on an annual basis.

JPP Evaluation Debriefing Seminar – Hanoi – June 2015 25

Recommendations• To the VBF:

– The VBF should reinforce internal communications, to enhance lawyers’ awareness of its actions and interest in supporting them.

– The VBF should prepare a medium- to long-term organisational development action plan, which could form a basis for diversification of funding.

• To JIFF:– The duration of NGO grants should be longer, and monitoring should

include more focus on outcome at beneficiary level; – MoJ concerns regarding administration and liquidation should be attended

to, by providing the necessary support.– NGO grantees should be encouraged to establish thematic groups at

national level to share information and contribute to judicial reform.

JPP Evaluation Debriefing Seminar – Hanoi – June 2015 26

Discussion• Two questions:

1. Do you agree with the evaluators’ assessment?

2. How could the programme have performed better?

Thank you