2019 gef project implementation report (pir) training undp ...€¦ · value of the pir process...
TRANSCRIPT
2019 GEF Project Implementation Report
(PIR) Training
UNDP Country Offices & Project Teams
Agenda
1. PIR process• Background on the PIR• 2019 timeframe and
requirements
2. Overview of 2019 PIR• Technical features• Tips for key PIR sections
& what’s new
3. PIR Quality• QA Background• Common quality issues• Tips for COs
Part 1: PIR Process
1. Introduction
2. Requirements
3. 2019 Timeframe
1. PIR Process: Intro to the PIR – the Ws
What?
• Annual monitoring of project progress toward its development objective/outcomes and implementation
Who?
• Mandatory input: Project Team, UNDP CO, UNDP-GEF RTA
• Optional input: GEF OFP, Implementing Partners, Others
When?
• Every year
• PIR process starts in early June
• PIR process ends in September
Value of the PIR Process
• Facilitates direct & detailed conversations among partners; enables the sharing of information and ideas.
• A PIR that provides a critical, accurate and transparent overview of project-level performance is valuable for the Project Board, other projects stakeholders, UNDP and the GEF.
• Reveals strengths & weakness in implementation; and areas for growth
• Critical assessment helps to ensure any required corrective actions are taken, and the appropriate support can be provided in a timely manner.
• Allows for a time of reflection on accomplishments; challenges; opportunities; risks and assumptions; Strategy and action plan for following year.
Why?
Country-level Roles in PIR ProcessPIR Role: Required to
contribute?
Responsibilities: Project
Function:
Notes:
Project
Manager/
Coordinator
Yes; is required
to contribute to
PIR
Manages the day
to day operations
of the project and
is closest the
project results.
Project
Execution
The Project’s Chief Technical Adviser
(CTA), Deputy Project
Manager/Coordinator, Gender
Specialist or other members of the
Project Team may log into the PIR
under the ‘Project
Manager/Coordinator’ role to provide
their input to the PIR as needed.
UNDP
Country
Office
Programme
Officer
Yes; is required
to contribute to
PIR
Provides oversight
and supervision
support to the
project.
Project
Oversight
Not applicable for DIM and many
regional projects.
GEF
Operational
Focal Point
Can contribute
to the PIR if they
wish, but this is
not required.
Represents the
government in the
country and serves
as the GEF OFP.
- Not applicable for DIM and many
regional projects.
Who?
2019 PIR – Country-level Roles in PIR ProcessPIR Role: Required to
contribute?
Responsibilities: Project
Function:
Notes:
Project
Implementin
g Partner
Can contribute
to the PIR if they
wish, but this is
not required.
Is the organization
that signed the
Project Document
for the project.
Project
Execution
Not applicable for DIM projects.
Other
Partners
Can contribute
to the PIR if they
wish, but this is
not required.
Represents other
Agencies working
with UNDP on
project
implementation.
- For jointly implemented projects, the
‘Other Partner’ is a representative of
the other Agency working with UNDP
on project implementation (for
example UNEP or the World Bank).
This is not to be used when another
Agency is the Implementing Partner
(they would contribute as the Project
Implementing Partner).
Who?
Roles in PIR ProcessPIR Role: Required to
contribute?
Responsibilities: Project
Function:
Notes:
UNDP-GEF
RTA
Yes; is required
to contribute to
PIR
Region-based
technical adviser
(RTA) accountable
for the overall
quality of the final
PIR.
Project
Oversight
-
UNDP-GEF
Regional
Programme
Associate
Yes; is required
to contribute to
PIR
Main contact point
with Country
Office for PIR
questions.
Project
Oversight
Responsible for keeping project’s
PIMS+ data and document up-to-date
so information is displayed correctly in
PIR.
Who?
2019 PIR Process - requirements
Yes PIR required in 2019:
• Projects with a project document signature date on or before 30 June
2018.
• All full sized projects (FSP) and medium sized projects (MSP) financed
by GEF/LDCF/SCCF/NPIF must submit a PIR each year of project
implementation.
No PIR required in 2019:
• Expedited EAs and MSPs* with a project budget under USD 1,000,000
• FSPs and MSPs do not need to submit a 2019 PIR if that project’s draft
or finalized Terminal Evaluation is posted in PIMS+ by 31 May 2019.
*Some teams may still require MSPs with a budget under USD 1,000,000 to complete a PIR. MSPs with a budget under USD 1,000,000
that have previously submitted the PIR will continue to submit the PIR.
2019 PIR Process – Timeframe
PIR Launch: First week of June
✓ Regional Team Leader sends to Country Office the unique hyperlinks to PIRs.
✓ Country Office shares hyperlinks with relevant Project Team members (CTA, Project Manager) and other stakeholders (Project Implementing Partner, GEF Operational Focal Point, Other Partners).
✓ PIR links available in *new CO Dashboard*
2019 PIR Process – Timeframe
First Draft of PIR: Date set by Regional Team
✓ The first draft of the PIR should be completed and sent by the Country Office to the UNDP-GEF team in the region for review.
✓ By this date, inputs should be provided by the Project Team, the GEF Operational Focal Point, the UNDP Country Office, and any other key national partners that would like to provide input to the PIR.
Part 2: 2019 PIR
1. Technical features
2. Updates
3. Tips for key PIR
sections
2019 PIR
• Same online PIR platform as 2018 on UNDP-GEF
PIMS+ database; COs can now access from PIMS+
CO Dashboard
• Tech support: [email protected]
• Use your mobile device to complete the PIR ☺
2019 PIR – Access
• Unique hyperlinks shared by Regional Team or
accessed from CO Dashboard (UNDP email required)
The Country Office’s list of projects that need to complete the 2019 PIR will appear at the top of the CO Dashboard.
Click the ‘FILL IN’ button to open the PIR.
Make sure to filter to the correct CO business unit.
2019 PIR – Access
✓ Log onto the PIR at any time during process.
✓ Multiple users can work on the same PIR at the same
time – everyone’s edits and changes will be captured.
✓ For users not accessing via CO Dashboard, the PIR login works the same as past years – users must enter their name and email, then select their role in the PIR process.
2019 PIR – Access
✓ Make sure to log in under the correct PIR role; all users can see all information entered in PIR but access rights differ based on PIR role.
2019 PIR
Let’s open the 2019 PIR online and take a look!
Feel free to open one of your own PIRs at this time and
follow along.
2019 PIR – Guidance Page
Please also read the guidance that is provided within each section of the PIR.
The first section of the PIR contains guidance text for each
PIR contributor.
2019 PIR – Features
View the project title and PIMS # of the PIR at the top of the screen.
View the PIR role which you are logged in as at the top of the screen.
2019 PIR – Features
Download a Word version of the PIR at any time by clicking ‘Word Export’
Attach files to the PIR (which will be saved in the UNDP-GEF PIMS+ database) by clicking on ‘File Library.’
2019 PIR – Features
Past years PIR Word Reports are accessible from the ‘Archives’ button
2019 PIR – Basic Data
Update Project Contacts; other data is for reference only, if
incorrect CO should notify Regional Programme Associate.
The only editable fields in the Basic Data tab are the name and email entry fields in the Project Contacts table at the very bottom (Project Manager; UNDP Country Office; GEF Operational Focal Point; Project Implementing Partner; Other Partners).
*New for 2019* Click ‘add new’ to insert the names of additional Project Managers/Coordinators, GEF Operational Focal Points and Other Partners (if applicable).
2019 PIR – DO Progress
In addition to cumulative results, Project Manager reports if
objectives and outcomes are track/off tracked/achieved
The Project Manager should always start the reporting by indicating the level of achievement of the target.
Show progress in target achievement incrementally, even if modest.
Project Manager sets progress for each objective & outcome as either: on track |off track|achieved – select the best option and be realistic!
2019 PIR – DO Progress
All inputs in this section should be cumulative from project start, not
annual.
Changes to indicators/baselines/targets cannot be made without official
approval from PSC and RTA.
If the objective/outcomes and corresponding indicators from the project
logframe/results framework are not accurate or complete, or were poorly
transferred to the 2019 PIR, please contact Regional Programme Associate.
Avoid these common mistakes in current year entries (Level at 30 June
20XX):
• No inputs, or minimally descriptive inputs
• Progress reported does not relate to the end of project target
• Information is not related to the indicator or inconsistent with each other or past years
• Information is not in the correct unit (i.e. different unit from baseline/target)
• Of different scope from baseline/target
2019 PIR – DO Ratings
• Progress summary of
objectives/outcomes is
displayed to inform DO
Rating.
• Provide a credible and
realistic rating.
• Include DO ratings
justifications as part of
Overall Assessment
If the Project Manager has selected off track for any objective/outcome in the DO Progress section then a Highly Satisfactory DO Rating cannot be selected.
2019 PIR – IP Ratings
• Only RTA and CO provide
IP rating.
• Detailed data on
financial performance
and milestones provided
to inform IP Rating.
• Include IP ratings
justifications as part of
Overall Assessment.
IP Rating cannot be satisfactory if: • Delivery rate GL expense against
the approved annual budget is less than 50%;
• Cumulative delivery is not on track;
• Measures to mitigate critical risks are not included in the PIR.
2019 PIR - DO & IP Ratings
• Project Manager, UNDP Country Office & UNDP-GEF RTA must all provide DO ratings and Overall Assessment every year
o all other partners’ inputs are optional but highly encouraged
• Only RTA and Country Office provide IP Rating
• Overall Assessment should fully justify/support/explain ratings and explain what is being (or will be) done to address problems
• Do not inflate DO and IP ratings
o (High PIR quality ratings depend on realistic DO and IP ratings
o A lower (i.e. unsatisfactory-range) rating is not ‘bad’; it signals transparent and responsible oversight and supervision
• Avoid these common mistakes:
o Rating is inconsistent with Overall Assessment (i.e. rating and assessment of rating do not match)
o Incomplete assessment (i.e. not comprehensive, do not outline next steps, etc…)
2019 PIR - Adjustments
*New items for 2019*
• The MTR management response
is provided for reference (if
available). The Country Office
must summarize the steps taken
to comply with the MTR
management response.
• Comments on delays in key
milestones are now required. The Country Office should also enter the dates of any project board meetings.
The Project Manager, Country Office and RTA must enter any comments on delays in meeting key project milestones during the reporting year, even to indicate ‘none.’
Country Office must enter critical risks only for the reporting year, from Atlas risk log, and measures to mitigate them.The total number of critical risks is used to calculate the overall risk rating of the project.
2019 PIR - Gender
The project’s Gender Analysis and Action Plan is displayed for reference; if no file is displayed the Country Office or Project Manager should upload the file or send to the Regional Programme Associate to upload via PIMS+.
*New items for 2019*
• The UNDP Gender Marker Rating
assigned in Atlas is displayed for
reference.
• There are new tick box fields to
indicate which of three results
areas the project is contributing to
gender equality.
• Question on whether women and
girls are direct beneficiaries has
been eliminated.
• There is a new question on direct
or indirect linkages between
project activities and gender-
based violence (info not shared
with GEF Secretariat).
The Country Office should update the Gender Marker Rating in Atlas if the results reported in PIR do not align with the rating.
2019 PIR - Safeguards
*New items for 2019*
This section has been
improved since its first
appearance in the 2018
PIR. It now contains four
multiple choice questions
and displays the project’s
SESP categorization.
It also links to any
safeguards management
plans in PIMS+.
A hyperlink to the project’s SESP and any management plans will be provided, if a SESP was prepared during project design; please download and open that file to see the originally identified social and environmental risks. That space will be empty for projects that did not prepare a SESP, but those projects should still complete this section.
2019 PIR – Communicating Impact
*New items for 2019*
• New SenseMaker survey
added (opens in a new
window).
• Question on project links and
social media expanded to
include KM activities per the
CEO endorsement/approval.
• New question on Project
Location Data. Please share
geo-referenced information
(including shapefiles) for
where the project
interventions take place.
Information on project links and social media reported in the 2018 PIR are displayed for editing.
2019 PIR – Communicating Impact - SENSEMAKER
*New for 2019*
• We are excited to work with
the UNDP Innovation Team
and the Executive Office to
introduce SenseMaker to the
PIR, a new way of sharing
project stories!
• We will not share any
SenseMaker stories or data
with GEF Secretariat. The
survey analysis is for internal
use only.
• Portfolio managers will have
access to an analyst tool of
SenseMaker results – stay
tuned!
Follow the steps and directions in the new window that opens and make sure to click ‘Submit Contribution to SenseMaker’ once complete (may need to scroll down the page)!
Every PIR contributor must complete the SenseMakersurvey! Survey will be available in French and Spanish and can be shared with project contributors who do not complete the PIR.
2019 PIR – Partnerships
*New items for 2019*
• Partnerships questions have
been changed from a text
response to yes/no.
• Question on South-South
cooperation is now included in
the Partnerships section.
• New question requiring update
of progress on the stakeholder
engagement plan.
• If any surveys have been
conducted related to
stakeholder engagement or
related areas please upload all
survey documents to the PIR
file library.
CEO endorsement/approval request is shared for reference to view the Stakeholder Engagement Plan.
2019 PIR – Overall Assessment
All PIR contributors
should complete the
Overall Assessment
section to summarize
progress and comment
on any issues that may
have affected
performance.
Other PIR Contributors’
Overall Assessment are
viewable.
Overall Assessment must include:• Full justification of ratings, including if
rating(s) differ from other contributors’ ratings;
• Explanation for any off track aspects of the project;
• Comments on implementation of previous year’s AWP;
• Assessment of project’s risks and risk management measures to be taken in the coming year.
• Plan to for addressing off track aspects.*See PIR Quality section for tips on
structuring the Overall Assessment*
2019 PIR – all sections are mandatory
Project Manager with
support of CO must
complete all sections of
PIR, including:
• DO Progress & Ratings
• Adjustments
• Gender
• Safeguards
• Communicating Impact
• Partnerships
• Overall Assessment
2019 PIR – Approve and Submit
• PM & CO must submit
PIR once their final
contributions have been
made.
• Submission status of PIR
contributors is displayed
for reference.
• All contributors receive
PIR Word Report after
submitting and when
RTA completes final
submission of PIR.
The RTA may revise the Overall DO and/or IP Rating if there are discrepancies in ratings such that the calculated overall DO and/or IP rating is much different than the RTA rating.
The Submit PIR button will only display once all required sections of the PIR have been completed (= Validation Successful). If required sections have not been completed the button will not appear and Validation Problem will be shown along with details on the required sections of the PIR that need to be completed.
Part 3: PIR Quality Assessment
1. Background; how the
PIR is assessed
2. PIR QA criteria
3. Common issues
4. Tips for COs
Background on PIR Quality Assessment
The PIR is a monitoring report that outlines progress achieved thus far in a given project and assessment by various project stakeholders of how the project is progressing and what should be done to improve performance.
PIR quality assessment examines the quality and accuracy of this monitoring role (i.e. as a proxy for oversight).
PIR QA is not an assessment of project performance:X Not an assessment of project outcomes or results based on the results framework;X Not an assessment of the project team or project manager.X Not a validation of the data or evidence provided in the PIR.
UNDP-GEF Directorate contracts an external independent reviewer to quality assess and rate each individual PIR every year.
Designed to foster continuous improvement in the quality of project monitoring and reporting.
Focus on transparent, credible and reliable evidence-based results reporting.
Access PIR Quality Ratings via the CO Dashboard
Access within CO Dashboard: https://co.undpgefpims.org/site/dashboardThen to go to: Reports>PIR Quality Assessment
PIR Quality Ratings and details going back to 2015 are available. View by region, CO or by individual project and see averages by CO against the target of 75% quality rated HS/S.
Make sure to filter to the correct CO business unit.
How is PIR quality assessed?
Five PIR quality Criteria
•Completeness•Balance•Consistency•Substance and reliability•Clarity
• The PIR quality assessor reviews the entire PIR and provides a rating against five quality criteria: completeness, balance, consistency, substance and reliability, and clarity.
• The DO and IP ratings and Overall Assessment are reviewed carefully to ensure they are substantiated by evidence and progress reported in the DO Progress, Adjustments, Gender, Partnerships, Communicating Impact, and Safeguards sections of the PIR.
• Not a rating of project progress or the project team!
Five PIR QA Criteria
Complete-ness
• PIR was submitted with all sections complete; provides comprehend-sive picture of project progress.
Balance
• Information provided was balanced, presenting realistic assessment of overall performance; positive and negative aspects of project progress are included.
Consistency
• PIR was internally consistent among different sections; supported by evidence.
Substantive-ness &
Reliability
• The overall assessments provided by the Country Office and RTA in particular substantiate the assessment of progress and performance and demonstrates strong oversight.
Clarity
• The information reported is presented in an understand-able and is clear.
PIR Quality Review– common issues
Complete-ness
• PIR was submitted complete; comments on the self-rating sections were completed and provided meaningful information.
Balance
• Information provided was balanced, presenting a complete picture; positive and negative aspects of project progress are included.
Consistency
• PIR was internally consistent among different sections.
Substantive-ness &
Reliability
• Information submitted provides sufficient context, key issues and factual evidence to substantiate the assessment of progress and performance
Clarity
• The information reported is presented in an understand-able and accessiblefashion.
Incomplete;Lacking full info
on progress
No info on weaknesses, challenges
Discrepancies across
sections
Rating not justified; weak
oversight
Info is unclear & of little value
Tips for improving PIR quality
➢ For projects that have not progressed as much as anticipated (such as projects doing their first PIR), sections left blank and lack of analysis of lower than
expected progress (even to explain no progress) are common issues leading to low PIR quality ratings.
➢ The fact that a given project has not undertaken many activities or is slow to become operational is not an excuse for not completing or explaining mandatory sections. The PIR should explain the reasons for delays or underachievement in
a transparent fashion.
➢ Most PIRs receive low quality ratings on clarity due to a lack of structure of Overall Assessment and the difficulty in following the reasoning behind the
DO/IP ratings given.
➢ Measures to manage or mitigate critical risks must be included in the PIR.
Tips for improving PIR quality
• The Overall Assessments of high quality PIRs have a clear structure that includes:
– full explanation/justification for the DO and IP ratings;
– discussion of each objective and outcome against end of project targets;
– Discussion of relevant issues/challenges/successes related to risk management, implementation (delivery rate, PMU, etc), delays, co-financing, key partnerships, gender, adaptive management;
– Concluding summary of overall progress and any final points, including a plan to address issues and challenges.
• Suggest to write the Overall Assessment in a Word file and then transfer to the online PIR.
Tips for improving PIR quality
It is strongly recommended to review examples of high quality rated PIRs:
• High quality 2017 PIR fully justifying HS DO Rating
• High quality 2017 PIR of project nearing closure
• High quality 2017 PIR of project with MS development progress
• High quality 2017 PIR of project with slow progress doing first PIR
• High quality 2017 PIR of project with unsatisfactory DO and IP
Tips for improving PIR quality
All inputs should be supported by solid evidence (e.g. Technical studies). Completed PIRs considered as ‘evidence’ for use during MTR and TE processes and for UNDP Project Quality Assurance.
Documentation of project-level progress and results, e.g. studies, monitoring reports, photos, and surveys, can (and should!) be uploaded into the PIR system.
Annual reporting for a given project can be made easier for the CO by using the same inputs in both the PIR and UNDP ROAR systems, as appropriate
Consult with stakeholders and partners during the preparation, to encourage dialogue and to support a balanced report
Note that all sections are mandatory!
Tips for COs on improving PIR quality
• Review Country Office, Project Manager and otherPIR roles and responsibilities task list in 2019 AMR guidance: https://undpgefpims.org/attachments/6187/216315/1726719/1739867/2019%20GEF%20Annual%20Monitoring%20Process%20Guidance.pdf
PIR Guidance available
2019 GEF Annual Monitoring Process Guidance (PIR, MTR, TE,
tracking tools):
https://undpgefpims.org/attachments/6187/216315/1726719/17
41026/2019%20GEF%20Annual%20Monitoring%20Process%20Gu
idance.pdf
Guidance Note on PIR Quality Assessment:
https://undpgefpims.org/attachments/6187/216315/1715903/17
21248/Guidance%20Note-PIR%20Quality%20Assessment.pdf
Yammer:
https://www.yammer.com/undp.org/#/threads/inGroup?type=in_
group&feedId=5264769024&view=all
PIR Uses at the Global Level
THANK YOU!!