introducing policy implementation and evaluation

Professional Certificate in Policy

Implementation and Evaluation

Ian “Ren” Rennie

INTRODUCING POLICY IMPLEMENTATION AND EVALUATION

1. Forward looking – takes a long term view, based on statistical trends and informed predictions, of the likely impact of policy

2. Outward looking – takes account of factors in the national, European and international situation and communicates policy effectively

3. Innovative and creative – questions established ways of dealing with things and encourages new ideas; open to comments and suggestions of others

4. Using evidence – uses best available evidence from a wide range of sources and involves key stakeholders at an early stage …

Professional Policy Making 9 Competencies

5. Inclusive – takes account of the impact on the needs of all those directly or indirectly affected by the policy

6. Joined up – looks beyond institutional boundaries to the Government’s strategic objectives; establishes the ethical and legal base for policy

7. Evaluates – builds systematic evaluation of early outcomes into the policy process

8. Reviews – keeps established policy under review to ensure it continues to deal with the problems it was designed to tackle, taking account of associated effects elsewhere

9. Learns lessons – learns from experience of what works and what doesn’t.

PROFESSIONAL POLICY MAKING FOR THE TWENTY FIRST CENTURY(1999)

Professional Policy Making 9 Competencies CTD

• “Implementing policy should never be separate from making it

• Successful outcomes depend on designing policy with clear objectives, creating realistic timetables and professional project planning

• Policy that is difficult to implement wastes time and money

• Effective delivery is particularly critical for the Government’s most important and high value projects, as this drives efficiencies and improves public services

• In the past, delivery of these major projects has too often been poor”.

UK Civil Service Reform Plan June 2012

• Be clear about the problem: High-level policy goals need to be matched with analysis of what problem government is trying to tackle and used to make good judgments on where to focus attention

• Work with the wider system: Policies are never implemented onto a blank canvas; they must compete for resources and attention with other national policies and local priorities – and can draw upon some of the assets that often already exist

• Stay close to implementers: Bringing others into policymaking is important, but once implementation begins central government also needs to keep strong links with where change is happening to understand how policies are working in the real and using regular stocktakes to keep up momentum.

Institute for Government Case Studies 2014

• Stay focused: Continuity is an essential ingredient of effective implementation. All of our case studies involved implementing over a period of many years and these long time-spans introduce significant risk to achieving policy goals.

• Use ministers to drive progress: While politics can add many complications to implementation, ministers play a crucial role in setting milestones and using regular stocktakes to keep up momentum.

• http://www.instituteforgovernment.org.uk/our-work/better-policy-making/improving-policy-implementation

Institute for Government Case Studies 2014 (2)

Different Systems FATALISM Control through processes (which are inherently unpredictable and fallible)

HIERARCHY Anticipative solutions, forecasting and management, enhanced authority and hierarchical ordering

INDIVIDUALISM Control through rivalry and choice, incentives to underpin market and individual choice processes (e.g. disclosure)

EGALITARIANISM Control through group processes, network style, participation

Four key control styles according to cultural theory (adapted from Baldwin et al 2011)

• Policy implementation in a top-down manner

• Policy implementation in a bottom-up approach.

Top-Down vs Bottom-Up

IMPLEMENTING WHAT?

• Civil servants are involved in design of policy or are affected in their service delivery by the policy design done by others

• “Though much policy advice is excellent, the quality of policy making is inconsistent and needs to be improved – too often policy advice draws from too narrow a range of views and evidence, and does not ensure that policy is capable of practical implementation”

• (Civil Service Reform Plan 2012).

Scoping Benefits for Policy Development

Is it:

• A supply issue

• A competition issue

• An information issue (e.g. to enable better choices)

• A cultural issue

• Etc?

The answer makes a big difference to which options will work!

Symptom vs Cause: The Real Delivery Problem

Implementation Insights – setting the goal • Have a clear, shared understanding of the problem and

prioritise outcomes • What problem needs to be addressed? Why does it

need to be solved now? • What are the key constraints (time, money, capacity,

legal etc) to tackling the problem? • How does the goal fit with other policy in the

department/government? • What is the right role for Government? • What constitutes success in terms of real world

impact? How will it be measured?

“Develop clarity about the purpose of the policy

• Be clear about the objective –– what issue(s) is/are being addressed

• Give clarity as to why this is/they are being addressed and what difference is intended

• Provide useable guidance that sets the scene for both the policy and the process”

‘INVOLVING THE FRONT-LINE IN POLICY-MAKING’ CABINET OFFICE

The Key

ASSESSING BUSINESS CASES ‘A SHORT PLAIN ENGLISH GUIDE’ HMT

5 Case Model

Strategic Context

• Organisational Overview

• Current Business Strategies

The Case for Change

• Spending Objectives

• Existing Arrangements

• Business Needs – current and future

• Potential Scope

• Benefits and Risks

• Constraints and Dependencies.

1) The Strategic Case

• Critical Success Factors

• Long listed Options

• Short Listed Options (including do minimum, three or four is the recommended number) Status quo; do nothing option (unless this is not credible)

• Economic Appraisals of Costs and Benefits with CBA Distributional Analysis (where relevant)

• Optimism Bias adjustment

• Risk Assessment

• Sensitivity Analysis

• The Preferred Option.

2) The Economic Case

• Procurement Strategy

• Service Requirements

• Charging Mechanism

• Risk Transfer

• Key Contractual Arrangements

• Personnel (TUPE) Implications

• Accountancy Treatment.

3) The Commercial Case

• Public Capital and Revenue Requirements

• Net Effect on Prices (if applicable)

• Impact on Balance Sheet

• Impact on Income and Expenditure Account (if applicable)

• Overall Funding and Affordability

• Commissioner Support (if applicable).

4) The Financial Case

• Programme and Project Management Methodology (PPM) and Structure

• Programme and Project Management Plans

• Use of Specialist Advisers Change and Contract Management Arrangements

• Benefits Realisation

• Risk Management

• Monitoring during implementation (proportionate)

• Post Implementation Evaluation Arrangements

• Contingency Arrangement.

5) The Management Case

STAKEHOLDERS

• The NAO report Modern Policy-Making:

• “Departments should make arrangements to engage implementers early so that the practicability of policies can be assessed. Involving implementers closely as policies are designed can help identify and manage risks of their effectiveness, secure ownership and commitment from staff, and identify practical solutions�”.

Implementers as Stakeholders

• “Front-line staff offer an effective route to meaningful evidence of the implementation and effectiveness of policy proposals. Given early involvement they offer a rich and sophisticated understanding of how their areas of work can be developed. As such, their input is an effective resource in ensuring that policies work first time. Their early involvement will provide informative evidence and give considerable insight to the development of policy, acknowledging a need to balance this view with those of other appropriate stakeholders..”

• ‘INVOLVING THE FRONT-LINE IN POLICY-MAKING’ CABINET OFFICE

Implementers as Stakeholders (2)

“Effective involvement of front-line staff requires:

• Clear objectives for the policy development and policy process

• Consideration of the front line as relevant to the policy in question

• A whole system approach to the involvement of the front line (and others)

• Proper resourcing and planning, an up front investment.”

‘INVOLVING THE FRONT-LINE IN POLICY-MAKING’ CABINET OFFICE

Implementers as Stakeholders (3)

• Define (who are they? Priorities?)

• Determine their expectations and concerns (communication)

• Decide what to do with stakeholder views (e.g. sharing with others in your organisation, letting stakeholders know what has happened)

• Decide systems for updating information (monitoring ongoing stakeholder views).

Stakeholder Issues

AGENCY COLLABORATION

• “Our analyses found that, whatever their size and complexity, all delivery chains are made up of only four basic types of links or relationships:

Delivery Chains

• Internal links, where one part of the chain directly manages another

• These links are often strong

• Effective delivery can be secured through levers such as internal performance and staff management

• Efficiencies, such as sharing services and improved asset utilisation, can also be identified

• Complex, outcome-focused PSA targets tend to have relatively few internal links in their delivery chains.

Delivery Chains 1

• Contractual or regulatory links, where one part of the chain defines through law and/or funding how another does its business

• These links can also be strong and effective at delivering outputs efficiently, as long as there is good contract design, good project management, and appropriate, strategic regulation

• There are often a number of links of this kind within complex delivery chains.

Delivery Chains 2

• Links of common purpose, where two bodies have parallel missions to work towards the common good and to make a positive difference to society

• Examples include the relationship between two government departments, or between a local authority and a local charity, each with a common interest

• These links proliferate in complex delivery chains and can be relatively weak, sometimes relying on good will alone to function.

Delivery Chains 3

• Links to the wider community, where one organisation has no formal authority over the groups and individuals with whom it wishes to work and is reliant primarily on persuasion to influence their behaviour to achieve targets

• Examples include attempts to persuade private sector firms to encourage car-sharing among their employees, or to support and encourage members of the public to stop smoking

• Yet these are often the weakest links in the delivery chain, with few positive incentives, and sometimes negative incentives, on those in the community to take part.

Delivery Chains 4

Implementation Insights

• Review evidence, appraise options, and decide how outcomes can best be delivered

• What policy levers and actions will best deliver the desired outcomes?

• What local, private and international evidence been considered in developing options?

• Why is the proposed solution the right one, as opposed to other options?

• What are the risks? How have unintended consequences been identified? What credible mitigations been identified?

• How can central elements of the solution be tested or piloted before you begin delivering at scale?

Ensuring Continuity

• Programme – proactively managing the process

• People – roles and responsibilities, awareness and education

• Processes – all organisational processes, including ICT

• Premises – buildings and facilities

• Providers – supply chain, including outsourcing

• Profile – brand, image and reputation

• Performance – benchmarking, evaluation and audit

Business Continuity Institute, 2003

• Technology

• Environment

• Geographical location

• Reward systems

• Rules and procedures

• Key organizational members

• Critical incidents.

Cultural Influences

Culture Affects Project Management

• Departmental interaction

• Employee commitment to goals

• Project planning

• Performance evaluation.

POLITICIANS AND BUREAUCRATS

• Leading sociologist Max Weber defined bureaucracy as an organisation with a hierarchy of paid, full-time officials who formed a chain of command

• A bureaucracy is concerned with the business of administration: with controlling, managing and co-ordinating a complex range of tasks

• Weber regarded it as dominant institution of industrial society (departments of state, political parties, businesses, education etc).

Bureaucracy

• Max Weber distinguished between types of action: – Affective or emotional action (stems from emotional

state at a particular time. Therefore hard to predict)

– Traditional action (based on established custom, done because always done that way. Therefore not necessarily responsive to current context)

– Rational action (goal orientated, systematic assessment of various options for meeting a goal and choosing option by established criteria). In many countries tends to be the public sector ideal.

Social Action Theory

• “Bureaucracy appealed as a way of preserving representative democracy whilst removing its worst features – instability, irrationality, and factionalism – from the day-to-day activities of governing

• Of course policy can never be separated from politics”

Bureaucracy- The Ideal (1)

• In government was hoped bureaucracy would:

– Provide a bulwark against the irrationalities of the electorate and their representatives

– Prevent organised interests taking control of public policy

– Permanent bureaucracy would divide politics from policy

– Provide a more scientific and evidence based approach to policy.

Bureaucracy – The Ideal (2)

• What are the risks and downsides of bureaucracy?

• How can these be addressed?

• (And how can the upsides be accentuated?).

Downsides and Risk of Bureaucracy?

• “quality assurance has only recently emerged as an occupation, or sub-field, of management. Today there are many complaints by public sector officials about the damaging effects of bureaucracy and red-tape created by this new form of regulation”

• Challenge of “driving up standards” of public service delivery versus extra red tape for staff involved (paper work, inspections, extra layers of checks etc).

‘The New Bureaucracy: Quality Assurance and Its Critics’ Max Travers 2007

Bureaucracy and Quality Management

Ministers and Special Advisers

• Influencing Ministers

• Using Special Advisers.

• “You want to take into account all the available evidence;

• but, at the end of the day, a Minister’s job, Parliament’s job is to reach a judgement as to whether or not a particular policy ought to be pursued …

• I strongly defend my right, as the Secretary of State, a Member of the elected Government to form a judgement as to what I think is the right thing to do

• and the Commons and Lords will decide.”

Reminder … the Political Context…

*Alistair Darling to a HoC Select Committee in 2006

PROJECT MANAGEMENT AND POLICY

• What is a project? • Defined start and end, specific scope, cost and

duration • A temporary endeavor undertaken to create a

unique product, service or result • A series of activities aimed at bringing about

clearly specified objectives within a defined time period and with a defined budget

(EU Aid delivery methods).

A project

• Project management was developed to save time by properly planning a project and considering all relevant factors which may affect its outcome

• The benefits have been proven - it saves time and money - and generates a more successful outcome …. if guidelines are followed.

Benefits of project management

• goal clarity and measurement • coordinated resources • risks will be identified and managed • increase the possibilities of time savings • increase the possibilities of cost savings • increase the possibilities of achieving the agreed

outcome • increase the possibilities to deliver projects

successfully.

How does project management benefit you?

• Stakeholder involvement • Executive management support • Clear statement of requirements • Proper planning • Realistic expectations • Smaller project milestones • Competent staff • Ownership • Clear vision and objectives • Hard working and focused staff.

Project success factors

The triple constraint

Projects process

Controlled start up

Controlled delivery

Controlled closure

Review

1. Obtain commitment from the business 2. Define the outcome 3. Define critical success factors (SMART) 4. Adopt processes for project management 5. Check staff skills and experience 6. Determine governance arrangements 7. Define the project scope ...

Controlled start up

Controlled start up

Controlled delivery

Controlled closure

Review

8. Determine the project approach

9. Plan for management of risk

10. Identify the dependencies with other projects

11. Determine whether procurement is required

12. Define the project organisation

13. Produce a project plan

14. Revisit scope

15. Define reporting arrangements.

Controlled start up

Controlled start up

Controlled delivery

Controlled closure

Review

1. Stage planning

2. Stage ends & decision points

3. Controls

4. Managing product delivery

5. Reporting against plans

6. Risk and incident management

7. Managing interfaces.

Controlled delivery

Controlled start up

Controlled delivery

Controlled closure

Review

1. Close the project.

Controlled closure

Controlled start up

Controlled delivery

Controlled closure

Review

1. Review and benefits realisation.

Review

Controlled start up

Controlled delivery

Controlled closure

Review

Identify scope and

stakeholders

Analysis

Information gathering

Identify sources of

information

Team Selection

Compare with: business strategy

business case

Use the information for more effective business operations

• Impact Assessment Toolkit http://www.bis.gov.uk/ia

• Treasury Green book (appraisal on vfm)

• https://www.gov.uk/government/publications/the-green-book-appraisal-and-evaluation-in-central-governent

• Magenta Book (Evaluation guide including analyst checklist) https://www.gov.uk/government/publications/the-magenta-book

• Orange Book (Risk management guide) https://www.gov.uk/government/publications/orange-book

• ‘Research Methods for Business Students’ Saunders et al

• ‘Making Policy in Theory and Practice’ Bochel and Duncan.

Sources of Help

QUESTIONS AND CLOSE