performance monitoring and evaluation
TRANSCRIPT
PERFORMANCE MONITORING AND EVALUATION
LEARNING EXCHANGE PROGRAM
Presented by
NIRMALA GOVENDER
Contents
Overview
Organisational Performance
Back to Basics reporting
Individual Performance
Annual Report
Overview
Performance Management is a legislative requirement in terms of chapter 6 of the Municipal Systems Act.
PME – Key Mechanism to monitor, evaluate, review and report on the Implementation of the Municipalities objectives
outlined in the Integrated Development Plan
‘If we are to improve our performance we have to reflect on what we are doing, what we are achieving against what we set out to achieve, and why deviations are occurring, or unexpected results
occurring. We cannot advance without making mistakes on the way, but we must evaluate and learn from our successes and our mistakes. Without this we cannot improve.’ Collins Chabane
The Constitution of the Republic of South Africa
Municipal Systems Act (2000)
Municipal Finance Management Act (MFMA)
Municipal Structures Act
Local Government White Paper (1998)
Regulations of the Systems Act
Performance Frameworks
Government Wide M & E Framework
Legislation and Policies
Monitoring, evaluation and reviewing constitutes a process to assess performance of the municipality. Monitoring and evaluation are crucial to:
• Ensure the implementation of plans and projects to ensure service delivery;
• Measure the development impact of plans/projects; and
• ensure the efficient utilisation of resources.
• Measures performance at an organisational and individual level
KPI’s and performance targets will be used as a yard stick to measure performance and to rectify shortcomings.
Monitoring is a continuous activity to measure KPI’s and performance targets. It will provide project information to management.
Monitoring includes the systematic collecting of information to enable management to evaluate whether project implementation is proceeding as it should.
Why PME is important
• you can not tell success from failureIf you do not measure results,
• you can not reward itIf you can not see success,
• you are probably rewarding failureIf you can not reward success,
• you can not learn from itIf you can not see success,
• you can not correct itIf you can not recognise failure,
• you can win public supportIf you can demonstrate results,
• Reinventing Government, 1992Adapted from Osborne and
Gaebler,
The Power of Measuring Results
ORGANISATIONAL PERFORMANCE MONITORING
CUSTOMERLooks at the organization through the eyes of its customers.
INTERNAL PROCESSESReports on the efficiency of internal processes and procedures.
FINANCIALMonitors the organization’s financial performance, which indicates whether the strategy, implementation and execution are contributing to the bottom line improvement.
LEARNING AND GROWTHReflects the commitment of the organization to grow and adapt to change.
BALANCED SCORECARD
INTEGRATED REPORT
(Kaplan and Norton, 1992)
National KPA BSC 8 Point Plan
Municipal Institutional
Development and
Transformation
Learning and
Growth
Creating a Platform for Growth,
Empowerment and Skills Development
Basic Service Delivery Customer Develop and Sustain the Spatial, Natural and
Built Environment
Creating a Quality Living Environment
Fostering a Socially Equitable Environment
Local Economic
Development (LED)
Financial Developing a Prosperous, Diverse Economy
and Employment Creation
Embracing Cultural Diversity, Arts and
Heritage
Municipal Financial
Viability and
Management
Financial Financially Accountable and Sustainable City
Good Governance and
Public participation
Internal and
Customer
Good Governance and Responsive Local
Government
IDP Organisational Scorecard
SDBIP (BUDGET)
Business Plans Project Plans
CITY MANAGER
DEPUTY CITY
MANAGERHEADS
DEPUTY HEADS/ SENIOR
MANAGERS
MANAGERSSUPERVIS
ORSOTHER STAFF
ORGANISATIONAL PERFORMANCE
INDIVIDUAL PERFORMANCE
National Treasury (2007)
RESULTS DRIVEN PERFORMANCE
IMPACTWHAT WE AIM TO CHANGE
ECONOMIC GROWTH/
WEALTH CREATION
OUTCOMEWHAT WE WISH TO ACHIEVE
JOB CREATION/SERVICE DELIVERY
OUTPUTWHAT WE PRODUCE OR DELIVER
COMPLETION OF PROJECTS
ACTIVITIESWHAT WE DO
PROJECTS/PROGRAMMES
INPUTSWHAT WE USE TO DO THE WORK
BUDGET/RESOURCES
Key
Performan
ce
Indicator
Annual
Target
2015/16
Target
as at 31st
March
2016
Status
as at 31
March
2016
Achieved/
Not
achieved
Performan
ce
Indication
Reasons
for non-
performanc
e
Measures
taken to
improve
performanc
e
What is it that
will be
measured
Annual
target for
the
financial
year
Accumulat
ed target to
date
What has
been
achieved
to date
Has the
target been
achieved?
A flag of
whether the
target has
been met
What is being
done to
improve
performance
where the
target has not
been metThe number of
consumer units
provided with
access to a
FREE basic
level of potable
WATER
253373 248573 257243 Achieved J
N/A
Availability of
Vehicles 95% 95% 92%Not achieved L
Delays in the
delivery of
spare parts.
Implementation
of 10 spare part
contracts with
the Original
Equipment
Manufacturers
(OEM).
Scorecard
Plan: 7
Plan Owner: City Manager
Operating Budget: R10 m for entire Plan
Capital Budget: R103.2 m for entire Plan
Programme Programme Driver
Project Unit Of Measure
Annual Target
Q1 -Sep
Actual for Q1
Reasons for Non-Performance
Measures Taken to Improve
Managingthe Informal Economy
John Smith Providesupport to the informal economy by upgrading BrookdaleMarket
% 100 29.90 13.65Delays in appointing contractor due to appeal on tender
1. Review project plan. 2. Additional support brought in.3. Work schedules reprioritised.
SDBIP
• the indicator should be accurate enough for its intended use and respond to changes in the level of performance.Reliable:
• the indicator needs to have a clear, unambiguous definition so that data will be collected consistently, and be easy to understand and use.
Well-defined:
• it must be possible to validate the processes and systems that produce the indicator.Verifiable:
• the usefulness of the indicator must justify the cost of collecting the data.Cost-effective:
• the indicator must avoid unintended consequences and encourage service delivery improvements, and not give managers incentives to carry out activities simply to meet a particular target.
Appropriate:
• the indicator must relate logically and directly to an aspect of the institution's mandate, and the realisation of strategic goals and objectives.
Relevant:
A good performance indicator should be:
KPI definition
• Compulsory submission
• Enhances understanding
• Submitted to Internal Audit, the audit committee & the AG
• Informs the audit
• No definition = KPI not SMART when audited
• Review when amendments are made
KPI Definition - 2016/2017
Type of KPI is your KPI an Input/ Output/ Outcome/ Impact, etc
Unit of Measure for KPI Number, percentage or ratio
Frequency of reporting i.e. quarterly or annually. If annual, why can this not be reported quarterly?
Background to KPI/ Rationale What is being measured and why, what impact will this have, why was this KPI chosen, Provides the logic behind the inclusion of the indicator as a reporting requirement, etc.
Indicator Origin - If this indicator is aligned to other sets of indicators, state its origin, particularly if part of an international reporting set
Legislation/ policies applicable to this KPI
MFMA, MSA, Labour relations
New KPI Definition Template.xlsx
• Specific
• Measurable
• Achievable
• Realistic
• Time Bound
Creating Targets
How are targets set and reviewed?
Targets are based on, access modeling, previous years actual status, research information or best practices.
Targets are reviewed every 6 months.
5 Year targets are set before the start of the 5 year cycle and are also reviewed annually.
Project plans/ Milestone definitions
• Compulsory submission
• Enhances understanding and informs the reader of the planned milestones
• Submitted to Internal Audit
• Informs the audit, particularly evidence required
• No project plan = Project & linked KPI not SMART when audited
• Review when amendments are made
List of projecttasks
Date of completion for task
% of task tothe total
Risks Dependencies
Project plan for numbers.xlsx
Project Plan
Reasons for non-performance / Measures taken to improve performance
Actions (measures) to improve performance – so by year end, target is met
Target not met - reasons for non-performance must be indicated –management action
The completion of the Measures to improve column is required by the Systems Act legislation.
Monthly & Quarterly reporting
Quarterly reporting
Report on time
Reasons for variance must be detailed.
Evidence
Measures to improve must be sufficient
MID-TERM
Main Criteria
Sub Criteria Definition
Usefulness
Measurability
Indicators are well defined and verifiable. Targets are specific, measurable and time bound
RelevanceIndicators relate logically and directly to the entity mandate and realisation
of strategic goals and objectives
ConsistencyObjectives, indicators and targets are
consistent between planning and reporting documents
Audit criteria (derived from performance management
and reporting framework)
Main Criteria
Sub Criteria Definition
Reliability
ValidityActual performance reported has occurred
and pertains to the entity
AccuracyAmounts, numbers and other data relating to
actual performance reported have been recorded and reported appropriately
CompletenessAll actual results and events that should have
been recorded have been included in the annual performance reports
Compliance with legislative requirements
ExistenceObjectives, indicators and targets must be
predetermined and performance information must be reported against them
Timelines
The annual performance report, together with the annual financial statements; must be
submitted for audit purposes to the auditors, within two months after the financial year end
Presentation
Performance information must be presented using the National Treasury guidelines.
Actual performance information in tables and other information included in the annual
report must be consistent.
Material differences between actual and planned performance must be explained
Main Criteria
Sub Criteria Definition
Common Findings - Audit findings are material – could lead to Qualified Audit
Objectives, indicators and targets not consistent between plan and annual performance report
Reasons for variances between planned and actual performance is not included
Logical link between objectives, indicators and targets do not exist
Changes to planned objectives, indicators and targets are not disclosed
Non-compliance to legislation
Non-timely submission of annual performance report
Indicators are not well defined
Evidence is not verifiable
Targets are not measurable
Common Findings
Lack of effective, efficient and transparent systems and internal control to provide sufficient, appropriate audit evidence to substantiate reported performance information
No supporting documents or insufficient supporting documents to support reported targets (validity)
The amounts, numbers and other data related to the actual targets did not agree to the supporting documents provided (accuracy)
The report had missing information, not everything related to the actual results and events where recorded (completeness)
Annual performance report was provided too late to conduct any audit work on reported information
Root causes
Lack of accountability and consequences for actions/inaction
Lack of monitoring by leadership
Inadequate formal planning for Predetermined Objectives
Misalignment between Strategy and Performance Reporting
Organisational structures not aligned
Structures not capacitated
Predetermined reporting not included in Performance Agreements
Poorly defined roles and responsibilities
Inadequate internal audit testing systems and data
Ineffective audit committee oversight
Oversight and support of Top Management and Council
Roles and responsibilities of all stakeholders must be understood
Buy-in from all Management and staff
Public Participation process critical
Skilled and committed Staff
Compliance to Legislation –critical deadlines/evidence
Efficient tools to be in place
LessonsLearnt
Lessons Learnt• must be developed, implemented and
reviewed annually.Policies, procedures
and frameworks
• must be SMART and have evidence to supportPerformance
targets & KPI’s
• Performance indicators/Data Elements must be well defined and verifiable.
Technical Indicator/ KPI Definition
• must be developed for KPI’s and must include monthly and quarterly targets.Project Plans
• must be alignedIDP/SDBIP/ Budget
and Scorecard
Lessons Learnt
• must be maintained, with definite actions to address audit findings
An audit action plan based on
the audit findings of the previous financial year
• must be convened to monitor performance, reasons for non-performance and measures to improve.
Quarterly performance
review sessions
Lessons learnt
• Keep things simple
• Try to keep requirements to a minimum
• Perform comparisons to previous reporting periods
• Attempt to understand operating environments
• Importance of communication
Planning
Resources
Procurement
Monitor
Ways of
improving
performance
Conduct feasibility
studies – needs,
costs, sustainability.
Know what you want.
Do you have the
resources to achieve
the target e.g. money,
HR
Create and submit a
procurement plan
Regular monitoring
and control of the
processes to achieve
targets
L
e
a
r
n
Learn from past
experiences
Reporting
Evidence
Targets
Accountability
Ways of
improving
performance
Ensure that progress
is reported on time.
No reporting = Not
achieved.
Collect & submit
evidence. No
evidence = Not
achievedSet stretch targets
that are still
achievable
Set targets in
consultation with staff
and create
accountability
New Developments
New developments
• Sustainable Development Goals
• Built Environment Performance Plan Indicators
• Municipal Chart of Accounts reporting requirements
• Back to Basics
BACK TO BASICS
APRIL-MAY 2014
5 PILLARS OF BACK TO BASICS Service Delivery – Creating decent living conditions
Good Governance & Administration
Public Participation
Financial Management
Institutional Capacity
AIM OF BACK TO BASICS• Serve our communities better - provision of
basic needs of communities
• Build a responsive, caring and accountable local government
• Ensure improvement in performance of local government
• Recognise & reward good performance
Individual Performance Management for Executives
ETHEKWINI MUNICIPALITY
Individual Performance Management System in EThekwini Municipality
PME - IPM System
• Senior Management on Executive Pay roll
HR - IPM System
• Middle Management and staff
MSA: Summary of legislation
Employment contract for Municipal Managers (MMs) and Managers Directly Accountable to MMs
• Performance agreement (PA),
o Individual Performance Plan
oPersonal Development Plan
IPM Framework IPM Framework for Senior Management of EThekwini
Municipality on Executive Payroll including MMs and Managers Directly Accountable to MMs• Incorporates the MSA, MSA Regulations, MFMA and Council
Resolutions
• Details an effective and efficient annual implementation process including
– Signing of PA’s
– Mid-term Reviews
– Annual assessments
IPM
By 2030 eThekwini will be Africa’s most caring and liveable city
Alignment of Processes (OPMS to IPMS)
IDP
SCORECARD
BUDGET
SDBIP
IPP
OPM
Content of IPPs
• Kpi’s from the Scorecard/SDBIP included in the Individual Plans
• Compulsory KPI’s
• Additional KPI’s
Structured Reviews
• 1st & 3rd quarter
– Verbal
• 2nd & 4th quarter:
– Recorded;
• Status report on performance
• Rating performance
Level Terminology Description
1 Unacceptable
performance
Performance does not meet the standard expected. The review/assessment indicates that the
employee has achieved below fully effective results for the performance criteria/indicators as
specified in the performance agreement/plan. The employee has failed to demonstrate the
commitment or ability to bring performance up to the level expected in the job despite management
efforts to encourage improvement.
2 Performance not
fully effective
Performance is below the standard required for the job. Performance meets some of the standards
expected for the job.
3 Fully Effective Performance fully meets the standards expected in all areas of the job. The appraisal indicates that
the Employee has fully achieved effective results against the significant performance
criteria/indicator as specified in the Performance Agreement/Plan
4 Performance
Significantly above
expectations
Performance is significantly higher than the standard expected in the job. The appraisal indicates
that the employee achieved above fully effective results against the performance criteria/ indicators.
5 Outstanding
Performance
Performance far exceeds the standard expected of an employee at this level. The appraisal indicates
that the Employee has achieved above fully effective results against the performance
criteria/indicators as specified in the Performance Agreement/Plan.
Rating Guide
Assessment results are compiled by PME and forwarded to CM
Assessment results of the CM are submitted to the MEC and the national minister for LG, within 14 days after the conclusion of the assessment
Assessments Results
Payment process of Annual Performance Bonus is undertaken by the CM and HR.
Performance bonuses are only be paid after :
• The completion of the annual assessments
• The adoption of the annual report
• On affordability by council.
Performance Bonuses
Dispute Resolution
Dispute on the nature of the employee's PA relating to;
Responsibilities
Priorities or objectives
Methods of assessment
Mediated within 30 days of receipt of a formal dispute from the employee
Dispute Resolution
Mediated by –
In the case of the City Manager; the MEC for LG in the province, or any other person designated by the MEC,
In the case of deputy City Manager, the mayor,
In the case of Executives; the City Manager,
Mediator’s decision shall be final and binding on both parties.
CHALLENGES RESPONSES
Late submissions o Notices (Emails and Memos)o CM Circulars
Non-submissions o CM Circularso No bonus payment
Confusing the process concepts o Training
Wrong templates o Annual dated templates
Incomplete and unsigned IPPs o Checklist
Constitution of prescribed panels o Guidelines on the onset
o Score sheet signatures
Securing availability of panel members o Scheduling along other meetings
o Teleconferencing
PURPOSE OF THE ANNUAL REPORT
ANNUAL REPORT
Provide record of the activities of the municipality and entities
in the Financial Year
Report on performance of service delivery and budget
implementation for the financial year
Promote Accountability
The Annual Report
• Recommends and submits Performance Information for approval
City Manager
• Own the IDP plans and responsible for ensuring Performance
Deputy City Managers – Plan Owners
• Monitoring and Evaluating Performance for the Org
Performance Monitoring and Evaluation Unit
• Plan Representatives
• Programme Drivers
• Project Managers
Senior Management team
RESPONSIBILITIES
Plan/Undertake and Manage
performance
• Participate on performance and KPIs
Citizens
• Recommend and Adopt performance reports
Exco/Council
• Review reports on performance quarterly
Support/Portfolio/Oversight Committees
• Reviews and provides oversight on performance
Audit Committee -External
• Audit and report on performanceInternal Audit
• Annual Audit and report on performance
Auditor General
• Review quarterly and annual PerfInformation
Prov/Nat COGTA/Treasury
RESPONSIBILITIES
Top Management and council to drive the system;
The Municipality’s Integrated Development Plan to be communicated Municipal-wide;
Channels of communication needs to be in place;
A clear understanding and appreciation of the value of having a performance measurement system;The development of employees identified through the Performance Monitoring and Evaluation process;
There should be a change management strategy in place;
The framework for achieving goals should also be SMART;
Review team goals at least once a month; and
Performance Monitoring and Evaluation should be considered as an on going process.
Key Issues Underpinning the Success of PME
Who contributes to the success of PME?
Citizens of eThekwini MunicipalityCouncil
Executive CommitteeCommittees of Council
City ManagerDeputy City Managers – Plan Owners
Senior Management teamPerformance Monitoring and Evaluation Unit
Plan RepresentativesProgramme DriversProject Managers
Internal AuditAuditor General
Audit CommitteeNational and Provincial COGTA and Treasury
PERFORMANCE OF THE CITY
Management
Councillors
Citizens
eThekwini =
VISION: Being Africa’s most caring and liveablecity by 2030
THANK YOU