Building Capacity for effective government wide Monitoring and Evaluation
Dr Mark Orkin and Mr Oliver SealeSAMEA Conference
Friday, 30 March 2007
Your Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and Development
2
PresentationStructure ♦ Strategic Objectives and Outputs (Mark Orkin)♦ Transformation to the Academy (Mark Orkin)♦ Capacity building for Monitoring and Evaluation (Oliver
Seale)
Objectives♦ To provide an overview of SAMDI’s current focus areas
and plot its future course.♦ To explore and engage with SAMDI’s capacity building
mandate for government wide monitoring and evaluation.
Your Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and Development
3
♦ Good Governance: Anti-corruption, Gender and disability, Batho Pele and Public Participation programmes.
♦ Capacity of the State: Local Government Strategic Agenda, Skills Assessment and Capacity Building programmes.
♦ Macro-organisation of the State: Single Public Service, Integrated Service Delivery and E-Government Service Delivery Projects.
♦ Transversal Systems: Integration of Planning and Government-wide Monitoring and Evaluation System.
Strategic Objectives and OutputsGovernance & Administration (G&A) Cluster Priorities
Your Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and Development
4
♦ Develop and administer a training framework for curricula and materials.
♦ Co-ordinate the provision of executive development programmes for the Senior Management Service.
♦ Develop and implement a quality management and monitoring system.
♦ Capacitate departments to identify their human resource development needs.
♦ Establish and maintain partnerships and links with national and international institutes and training providers.
♦ Arrange customised training programmes to support foreign policy on the African Union (AU) and the New Partnership for Africa’s Development (NEPAD).
Strategic Objectives and OutputsSAMDI 2007/08 to 2009/10
Your Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and Development
5
Year-on-Year Comparison
0
2000
4000
6000
8000
10000
12000
14000
Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar
PTD
's
2005/06 2006/07
Strategic Objectives and Outputs Training Statistics: 1 Apr. 2006 – 28 Feb. 2007
Your Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and Development
6
Strategic Objectives and Outputs PTDs* delivered in Provinces (to end-Jan. 2007)
KZNMpumalangaLimpopoWestern CapeEastern CapeNorthern CapeFree StateGautengNorth WestNational
Distribution of Government Employees per Province
Distribution of PTDs per Province
14% 18%5%
5%10%
14%6%
2%11%5%
2%3%
5%3%
10%18%
6%
31%
20%
12%
* PTDs = Person Training Days
Your Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and Development
7
♦ Support JIPSA (Joint Initiative On Priority Skills Acquisition) policy formulation and training.
♦ Incubate AMDIN (African Management Development Institutes’ Network) and DRC (Democratic Republic of Congo).
♦ Contribute to the ASGI-SA (Accelerated and Shared Growth Initiative for South Africa) through concentrated public sector human resource development activities and operations.
♦ Transformation to the Academy – refer detail slides.♦ Capacity building for Monitoring and Evaluation – refer detail
slides.
Strategic Objectives and OutputsFocus areas for 2007/08 and beyond
Your Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and Development
8
Transformation to the Academy SAMDI: Need for a paradigm shift♦ R1 billion p/a spent in departments but 43% of staff in
provincial departments reported no training in 2006.♦ International benchmarks suggest at least 5 days training per
annum: For approx. 250,000 middle and junior managers requires 1,25
million PTDs p.a.; Allowing for 60% of training already occurring in departments still
requires 0,5 million PTDs p.a. ♦ For induction, staff turn-over is 120 000 people p.a. requiring
another 0,2 million PTDs.♦ Thus, the total-demand driven requirement is 0,7 million
PTDs: nearly 10 times SAMDI’s present output!
Your Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and Development
9
Transformation to the Academy Vision and activities ♦Three “mantras”
Provision to facilitation.Competition to collaboration.Selective coverage to massification.
♦First main stream of activityExecutive development programmes for SMS.Entrant, lower and upper SMS: programmes, courses and events. In collaboration with universities and counterparts.
♦Second main stream of activity“Massified” management training for junior and middle managers.Training frameworks of curriculum and materials in conjunction with
provincial academies and DPLG; Monitoring and Evaluation to regulate providers; The induction programme for new entrants at all levels.
Your Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and Development
10
Transformation to the Academy 2006/7 SAMDI outputs: basis of new approach
0
5000
10000
15000
20000
25000
30000
35000
MDT ELD HRDT SCM HRMT FPMT SDT ID
MLD MLD MLD CRMT CRMT CRMT CMSDI CMSDI
Pers
on-T
rain
ing
Days
(PTD
s)Univ. etcContractorSAMDI
2006/07
MDT- Management Development Training; ELD - Executive Leadership Development; HRDT- Human Resource Development & Training; SCM: Supply Chain Management; HRMT - Human Resource Management Training; FPMT - Finance & Project Management Training; SDT - Service Delivery Training; ID - Institutional Development
Your Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and Development
11
Transformation to the Academy ENE training spend in national departments
0.00
1.00
2.00
3.00
4.00
5.00
6.00
Justi
ce an
d Con
. Dev
.
Arts ab
d Cult
ureDEAT
Educa
tion
Social D
evelo
pmen
t
DPLG
Labo
ur
Health
Land
Affa
irs
Transpo
rt
Agricu
lture
Parlia
ment
Safety
and S
ecuri
ty
Home A
ffairs
Public
Works
Water A
ffairs
and F
orestr
y
Housin
g
Perc
ent
Tr/Curr %
Tr/Pers %
Your Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and Development
12
Transformation to the Academy Learning framework: tentative harmonised modules
Performance levels
Senior
Middle
Junior
Supervisor
Fina
nce
Peo
ple
Pro
ject
s
Cul
ture
Info
rmat
ion
Oth
er
Fina
nce
Hum
an R
es.
Sup
ply
Cha
in
Info
rmat
ion
Oth
er
Oth
er
Imm
igra
tion
Pen
sion
s
Indu
ctio
n
Generic competencies
Functional competencies
Sectoral competencies
Your Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and Development
13
Transformation to the Academy Projects for Internal Task Teams
1. Audits of junior and middle management courses
2. Planning and implementing the training framework
3. Enhancing the monitoring and evaluation system
4. Streamlining accreditation processes
5. Planning for massified induction programme
6. Preparing for service provider mobilisation
7. Planning executive programmes for SMS level
8. Recasting delegations and policies
9. Scoping operational system for outsourced training
10.Conceive a knowledge management system
Your Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and Development
14
Transformation to the Academy Recap and way forward
♦ Executive development programmes.♦ Learning framework for massified middle and junior
management learning.♦ Curricula and material development, guality assurance and
accreditation.♦ Provider and user relations; M&E of large-scale provision ♦ Provincial infrastructure.♦ Research capacity and networking.♦ Continental support for Management Development Institutes,
international relations.♦ Impending restructuring process.
Your Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and Development
15
Capacity building for Monitoring & Evaluation BackgroundAims and objectives
The aim of the system is to contribute to improved governance and enhanced effectiveness of public sector institutions.
The system aims to collect, collate, analyse and disseminate information on the progress and impact of programmes.
Result Areas Accurate information on progress in the implementation of public
sector programmes is updated on an ongoing basis; Information on the outcomes and impact achieved by
government is periodically collected and presented; The quality of monitoring and evaluation (M&E) practices in
government and public bodies is continuously improved.
Your Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and Development
16
Capacity building for Monitoring & Evaluation Provincial needs analysis: example of feedback (A)
1. Little coherent or articulated strategy in provinces, though expenditure on expensive systems to collate M&E data.i. What would a coherent strategy need to contain? ii. What is an articulated strategy? What type of links
are we looking for? iii. What systems are there? What do we mean by a
system?iv. What data are there? How were they obtained? What
is the quality of this data? v. What collation is taking place? How? Can a system
collate data?
What are the
implications for training?
Your Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and Development
17
Capacity building for Monitoring & Evaluation Provincial needs analysis: example of feedback (B)
2. Monitoring programmes are just about collecting data, very little analysis and feedback given.
i. What data, why and how are they being collected?
3. Alignment of plans doesn’t existi. What planning does take place and how? ii. How is M&E incorporated into planning?iii. What do we mean by alignment why do we
need it? iv. Is alignment always possible and necessary?
What are the
implications for training?
Your Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and Development
18
Capacity building for Monitoring & Evaluation Provincial needs analysis: example of feedback (C)
4. Planning without indicatorsi. What type of indicators do we mean and how
should these be developed?ii. What indicators do exist and how are they
measured?iii. How are they decided on?
5. Lack of or poor baseline datai. What baseline data do exist and do we evaluate
it?ii. What type of baseline data are required?iii. How are these presently obtained?
What are the
implications for training?
Your Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and Development
19
Capacity building for Monitoring & Evaluation Conceptual framework for training
1.1. DescriptionDescription2.2. Existing data basesExisting data bases3.3. Data collection methodsData collection methods4.4. Baseline data Baseline data
1.1. What will be done (strategy)What will be done (strategy)2.2. Why will it be done (policy)Why will it be done (policy)3.3. How will it be done (operations)How will it be done (operations)4.4. Indicators and criteria (how to measure)Indicators and criteria (how to measure)5.5. When (timeframes)When (timeframes)
MonitoringMonitoring
Existing situationExisting situation New project or New project or programmeprogramme
PlanningPlanning
1.1. System to be used (MIS)System to be used (MIS)2.2. Indicators Indicators 3.3. MethodsMethods4.4. Baseline dataBaseline data5.5. Inputs Inputs 6.6. TrackingTracking
i.i. ProcessesProcessesii.ii. ActivitiesActivities
7.7. Interventions and modificationsInterventions and modifications8.8. OutputsOutputs9.9. OutcomesOutcomes
EvaluationEvaluation
1.1. System to be used (EIS)System to be used (EIS)2.2. IndicatorsIndicators3.3. MethodsMethods4.4. Baseline dataBaseline data5.5. CriteriaCriteria6.6. AssessmentAssessment7.7. ProcessProcess8.8. ImpactImpact9.9. Lessons learnedLessons learned10.10. FeedbackFeedback
Your Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and Development
20
Capacity building for Monitoring & Evaluation Training principles for various levels
Principles
1. Understanding of basic principles of M&E
2. Applying principles to a specific project
3. Applying principles to a programme
4. Applying principles to overall management in departments
5. Applying principles across departments/provinces
6. Actually performing evaluations
Levels
Basic for general users of information Basic for project managers Intermediate for programme managers Advanced for executive managers Advanced for CFOs and DDGs Specialist technical training for M&E staff
Your Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and Development
21
Capacity building for Monitoring & Evaluation Target Audiences1. Users
i. Political heads and parliamentarians (incorporated into report-backs to portfolio committees)
ii. Accounting officers (DGs)iii. Executive managers and managers in govt departmentsiv. Users of the service or the information outside government
2. Producersi. Programme managersii. Project managersiii. Operations staffiv. Participants
3. M&E staff in national and provincial departments
Your Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and Development
22
Capacity building for Monitoring & Evaluation
Examples of current provisionInstitution Programme Duration Level Content
University of Stellenbosch
Diploma in Monitoring and Evaluation Methods
1 year Post Graduate
♦ General principles & paradigms ♦ Clarificatory evaluation♦ Process evaluation & programme monitoring♦ Data collection methods♦ Statistical and qualitative methods♦ Impact assessment designs
University of Cape Town
Workshop on Monitoring and Evaluation
1 week All ♦ General principles♦ Measuring Public projects
Regenesys Short course in Monitoring and Evaluation
3 days NQF 4 ♦ Monitoring and evaluation concepts♦ Results-based management♦ Concepts of outcomes♦ Project objectives and indicators♦ Monitoring and evaluation system♦ Team performance improvements♦ Performance standards ♦ Risks and the impact thereof♦ Success and failure factors♦ Project evaluation reports
Your Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and Development
23
Capacity building for Monitoring & Evaluation Strategy and Plan of Action♦ Progress Report
Terms of reference developed for Task Team. 15 workshops on M&E for programme and project management
(340 officials). Initial needs analysis on provincial M&E capacity. Consultation with key internal and external stakeholders.
♦ Plan of Action Research on M&E training needs - SMS, MMS, JMS,
practitioners (March ‘07). Determine current providers of M&E - HEIs, privates, NGOs etc.
(March ‘07). Undertake training needs analysis for M&E (May ‘07). Develop M&E training programme (Sept ‘07); roll-out (Nov. ‘07).
Your Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and DevelopmentYour Partner in Improving Service Delivery through Training and Development
24
Siyabonga
Thank you
Rolivhuwa
Dankie
Nakhensa
Re a leboga