the performance measure problem…

Paul D. Epstein, Epstein & Fass Associates 1

The Performance Measure Problem…Dept. Head: “So we are cleaning sewers that are prone to be clogged?”Prgm. Mgr.: “What do you mean? You want us to clean sewers that are dirty? I don’t think so. We are rated on how many miles of sewers we clean. We take great pride in the number of miles. We clean a lot of sewers.”Dept. Head: “Are you telling me that we clean onlyclean sewers?”Prgm. Mgr.: “Well, yes, because if we clean a dirty sewer it’s going to slow up productivity.”

2

Strategic Planning & Performance Measurement

Department of Administration

Research, Planning, and Grants

Course Developed by Julie Butler

Presented by Maud Naroll

3

Class Goals

Introduce concepts of Strategic Planning & Performance Measurement.

Students leave with practical tools to create their own strategic plans and outcome-based performance measurements.

4

People and their managers are so busy trying to do things right that they hardly have time to decide if they’re doing the right things.

- Stephen R. Covey -

5

Strategic Planning…What

An organized, documented method of determining what an agency hopes to accomplish and how it will accomplish it.Future-oriented.Value from the process.

6

Strategic Planning…Why

Management tool.Provides direction for agency personnel.Guides resource decisions.Provides justification for budget requests.

7

Strategic Planning

Answers 3 basic questions: Where are we

now? Where do we want

to be? How do we get

there?

8

Strategic Planning - First things first…Support from the top.Team-approach. Management &

staff. Program & fiscal. Data administrator

Define the scope. Department,

division, bureau, section.

Time commitment.

9

Components of a Strategic Plan -- Overview

Will present one way of developing plan Has worked with a couple dozen

agencies

No single correct way to plan Don’t feel is rigid cookbook

Suggestions welcome

10

Components of a Strategic Plan -- Overview

VisionMissionPhilosophy(External/Internal Assessment: Customers and SWOT)Goal(s)StrategiesObjectivesPerformance Measures

11

Components of a Strategic Plan

Vision Statement Describes the

ideal future state What we aspire to Should inspire

Big, global, broad

12

Sample Vision Statement

Employment, Training and Rehabilitation

To be Nevada’s First Choice to connect businesses and job seekers

13

Sample Vision Statement

Levi Strauss & Company

People love our clothes and trust our company.

We will market and distribute the most appealing and widely worn apparel brands.

Our products define quality, style and function.

We will clothe the world.

14

Components continued…

Mission Statement

The foundation of the strategic plan. Defines the purpose of the agency & how it

will work to achieve the vision. Describes what the agency does for whom. Keep it simple and easy for all to remember. Ask: Why was the agency created?

15

Sample Mission Statements

Nevada Department of Personnel:

To provide and retain a qualified state workforce that serves the citizens of Nevada.

Carson Tahoe Regional Healthcare:

To enhance the health and well being of the communities we serve.

16

Tool for Mission, Goals, Strategies, and Objectives

To help draft your agency’s mission, goals, strategies & objectives, consider the following:

The purpose of the ___________________(program) is

to provide/produce _______________ (service/good)

to ______________ (whom)

so that they can/in order to _______________ (planned benefit).

17

Class Exercise

For one of the agencies in class, fill out this form.

Optional: craft a short, simple mission statement.



to ______________ (whom)


18

Individual Exercise

For your agency, fill out this form.

Optional: craft a short, simple mission statement.



to ______________ (whom)


19

Class Exercise

For one of the agencies in class, write a vision statement

20

Individual Exercise (Optional)

For your agency, write a vision statement

21


Philosophy/Values How we will

act/behave as we work to achieve the vision.

Should guide the agency in its daily interactions with staff and customers/clients.

22

Sample Philosophy Statements

Budget and Planning Philosophy:

We will strive for excellence in an ethical, accountable and responsive manner. We will treat our fellow staff members, customers, and stakeholders consistently, fairly, and professionally. We are stronger as a team than individually. We will share our individual knowledge and experience to help each other. Carson Tahoe Regional Healthcare:

•Putting Patients First

•Treating Everyone with Dignity and Respect

23

Sample Philosophy Statement

Nevada Department of Corrections, Prison Industries Division

All staff will act in a fashion that presents to the inmates an ethical and moral model that, were they to emulate, they would never return to prison.

24

Class Exercise

For one of the agencies in class, write a philosophy statement

25

Individual Exercise (Optional)

For your agency, write a philosophy statement

26

Table Exercise (Optional)

Fill out the mission for your agency (if not done already).Examine (or create) your agency’s current vision, mission, and philosophy statements.Compare to checklists.Do the statements need to be revised?Presentations to the class.

27

External/Internal Assessment: Defining the Customers

InternalExternalStakeholders

Who are they?What do they need?Are we meeting their expectations?How will we know?

28

Class Exercise

For some agencies in class, identify customers, their expectations, and whether they’re being met

29

External/Internal Assessment: SWOT

Think about workforce issues: Retirements Career Ladders Cross-covering

eliminated positions

Strengths Weaknesses

Opportunities

Threats

INTERNAL

EXTERNAL

30

External/Internal Assessment: Examine the Data

Where have we been? Where might we end up if it’s “business as usual?”Ask lots of questions. More customers now than before? Same # of staff or turnover? Do we have outdated procedures,

processes or systems? Do our customers perceive a

problem?

31

External/Internal Assessment: Who are we?

What do the laws, regulations, and policies say about what we do? Should the laws/regs/policies be changed?

Does anyone else do what we do?If so, who are they?Do we coordinate with them?

32

External/Internal Assessment: What is affecting us?

STEEPL

SocialTechnologicalEnvironmentalEconomicPoliticalLegal

33

Class Exercise

Write one each: Strength Weakness Opportunity Threat

Write one STEEPL item

Strengths Weaknesses

Opportunities

Threats

INTERNAL

EXTERNAL

34


Goals Broad description

of WHAT the agency hopes to achieve over the long term (3+years).

Start with an action verb

35

Sample Goals

Streamline the state recruitment process. (NV Dept of Personnel)

Improve the quality of health care in Nevada. (State Health Div – Bureau of Licensure & Certification)

Secure site, build, and occupy new Las Vegas facility. (Motor Pool)

36


Strategies The methods/actions the agency will

take to achieve the goal.

The “HOW” part. Specifies an Activity not a Result.

May be several strategies per goal.

37

Sample Strategies

Open recruitments on a daily basis versus a weekly basis (NV Dept of Personnel).Educate health care providers about the reasons health care facilities are found deficient through brochures, conference calls, and the internet. (Health Div – BLC)Aggressively work with State Lands, Buildings and Grounds, and the developer. (Motor Pool)

38

Brainstorming Strategies

Cross-train staff to do multiple tasks? Reclassify staff? Review procedures & streamline? Modify computer system? Explore more internet-based

solutions? Stop doing some tasks?

39

Prioritize and Prune Strategies

After writing the plan, go back through strategies.

Is the list feasible to accomplish in the next several years?

Do strategies need prioritizing? Pruning?

40


Objectives Specify the target…how much of the

desired result by when. May be more than one objective for each

goal. If an objective is not measurable, then

it’s a goal.

41

Objectives continued…

Objectives should be SMART

Specific Measurable Action-oriented Realistic Time-Specific

42


Start with an action verb

Example: Occupy new Vegas building before January 2007. (Motor Pool)

43


Determine a “reasonable” objective Set a date by which to achieve your objective.Example: 5% reduction in average wait time by October 2010 and 10% reduction by June 2011.

44

More Sample Objectives

Projects in BA 1325 completed within budget by June 30, 2007. (Dept Admin IT Division)Average fleet miles per gallon five percent higher than January 2006 level by January 2009. (Motor Pool)

45

But Sometimes Objectives Are…

Yes or No

Occupy new Vegas building before January 2007. (Motor Pool)

Implement our revised office procedures by October 2014.

Update all employee work performance standards by December 2015.

46

Impact on the Budget

What is the cost associated with each strategy?Can we pay for this?

What can we do for no cost or low cost?

Move money around?

How critical is this?Contact your fiscal staff for help.

47

Class Exercise

Examine or create an agency’s goals and related strategies & objectives.

48

Table Exercise (Optional)

Examine (or create) one of your agency’s goals and related strategies & objectives.Compare to checklists.Revise if necessary.Presentations to the class.

49

Performance Measurement

It all starts with the Strategic Plan

Paul D. Epstein, Epstein & Fass Associates 50

The Performance Measure Problem…Dept. Head: “So we are cleaning sewers that are prone to be clogged?”Prgm. Mgr.: “What do you mean? You want us to clean sewers that are dirty? I don’t think so. We are rated on how many miles of sewers we clean. We take great pride in the number of miles. We clean a lot of sewers.”Dept. Head: “Are you telling me that we clean onlyclean sewers?”Prgm. Mgr.: “Well, yes, because if we clean a dirty sewer it’s going to slow up productivity.”

51

Performance Measurement Is:

A method of systematically and objectively tracking the agency’s progress toward achieving its mission & goals.

52

Performance Meas. Is Not

Substitute for Program Evaluation

The Magic Cure Can tell you the “what” but not the

“why”. You have to dig deeper to learn the

meaning behind the numbers.

53

Why Measure Performance?

54

Why Measure Performance?

Management Tool To determine if you’re getting where you planned. To improve management processes & practices. To improve quality & efficiency. To recognize & reward success. To ensure accountability. To communicate results & improve agency image. To justify budget request & guide resource

decisions.

Because it’s the law. [NRS 353.205(1)(b)]

55

NRS 353.205(1)(b)

…Part 2 must include a mission statement and measurement indicators for each program…..

56

Using PM’s to Manage

Are we doing the right thing?

Are we doing it well?

Are we investing our resources to get the best results?

Are we getting where we want to go?

57

Linking to the BudgetThe Strategic Plan (SP)

is the foundation of the budget request. If it’s not in the

SP, then it shouldn’t be in the budget.

58

Linking to the BudgetThe Performance

Measures tell the budget “story.” This result at this

cost. Justify changes to

your program(s). Illustrate

consequences of cuts.

Budget & Legislature review them.

59

Measures in the BudgetMeasures now in the Priorities and Performance Based Budget (PPBB)

At the activity level

60

Measures in the BudgetExample

Division: Fleet Services

Activity: Short-term Assigned Vehicle Management

Performance Measure: Rental Rate as a Percent of Commercial Rate

FY 2012 actual: 70 %

61

Why Don’t We Like Performance Measures?

62

Why Don’t We Like Performance Measures?

Time & cost of collecting the data.Fear of being held accountable.Outdated information systems and procedures.Difficulty in making fair comparisons and finding appropriate benchmarks.Can’t say why results are occurring.Staff turnover.Don’t have complete control over an outcome.

63

Types of Measures

InputOutputOutcomeEfficiency Timeliness

QualityPopulationCaseload

64

DefinitionsInput: measures the resources going into making a product or providing a service Not necessarily useful in budget Example:

Money – “We got a $100,000 grant for it.” Staff - “ We have 5 employees devoted to it.” Equipment – “We bought a new server for it.” Raw materials – “We used 10 cubic yards of

cement for it.”

65

Definitions continued

Output: measures what comes out of the agency’s efforts (“widget counting”)Number of:

Miles of sewer cleanedMiles of road resurfacedClients servedApplications processedMeetings held

66


Outcome: measures the result of your efforts. Did you have an impact? What changed in your environment in

part because of your efforts?

67


Outcome: measures the result of your efforts. Percent reduction in vaccine-

preventable diseases Percent of clients employed 6 months

after completing program

68


Efficiency: ratio of outputs or outcomes produced to inputs usedNumber of

Customers served/employeeCases managed/employeeMiles of road paved/paving

machine

69


Efficiency - Timeliness:

average DMV wait time average turnaround time to

process form

70


Quality: measures quality of the services provided or goods produced

Customer satisfaction surveysOpinion pollsLetters or phone calls of

praise/complaint

71


Population: Number of people or entities an

activity might potentially serve

Number of:Nevada children aged 5 - 18Businesses in Nevada with

employeesNevadans eligible for Medicaid

72


Caseload: Number of people or things the activity serves or funds

Number of: K-12 students whose education is

supported with state funds Businesses paying unemployment

insurance taxes Medicaid enrollees

73

Developing a MeasureStart with the Strategic Plan.The Vision defines where you want to go.The Goal defines what you want to accomplish (broadly).The Objective defines your target (what you’re aiming for).The value of the Performance Measure tells you whether or not you met your objective, whether your strategy worked, whether you’re on the way toward achieving your goal.

74

Developing a Measurement

Focus on OUTCOMES, not OUTPUTS Example: agency

held 3 grant writing workshops last year (output).

Agency received better grant applications (outcome).

75

Common Outcome Measures

Behavior – did customer’s behavior change (in part) because of your agency’s efforts? Example: fewer

adults smoke after cessation campaign

Condition – did the “state of things” change (in part) because of your agency’s efforts? Examples: cleaner

water, better roads, more employment

76


Knowledge: does the customer know more about “X” after contact with your agency? Example: % of

Nevadans who know the illegal per se limit for Driving Under the Influence in Nevada

Attitude: does the customer believe your agency’s efforts will make a difference? Example: % of

prgm. participants who feel confident agency’s job training will help them get a job

77


Skills: did skill levels change (in part) because of agency’s efforts? Example: Percent

of 4th graders reading at 4th grade level

Values/Ethics: does customer know right from wrong after contact with your agency? Example: Percent

of Nevadans who believe leaving scene of traffic accident in which they were involved is wrong

78

Performance Measure Example: Buildings and Grounds

Goal = Fulfill agencies’ needs for leased space

Objective = Reduce average time from initial request on lease until lease sent to Budget by 10% by July 2009

Possible Performance Measure = ?

79

Perf. Measure Example: B&G

Goal = Fulfill agencies’ needs for leased space

Objective = Reduce average time from initial request on lease until lease sent to Budget by 10% by July 2009

Performance Measure = Average time from initial request on lease until lease sent to Budget

80

Types of Measures



What type of measure is

Average time from initial request on lease until lease sent to Budget

81

Performance Measure Example: Employment, Training, and Rehabilitation

Goal = Provide convenient, accessible and equitable services and opportunities to all customers

Objective = Increase the use of Interactive Unemployment Insurance applications for DETR services by 5%


82

Performance Measure Example: Employment, Training, and Rehabilitation

Goal = Provide convenient, accessible and equitable services and opportunities to all customers

Objective = Increase the use of Interactive Unemployment Insurance applications for DETR services by 5%

Possible Performance Measure = Percent of unemployment insurance clients using interactive applications

83

Performance Measure Example: Carson-Tahoe Regional Healthcare

Mission = To enhance the health and well being of the communities we serve. Possible Performance Measure = ?

84

Performance Measure Example: Carson-Tahoe Regional Healthcare

Mission = To enhance the health and well being of the communities we serve. Possible Performance Measure = Deaths per 100,000 people under 65 in our primary service area

85

Types of Measures


Quality

What type of measure is

Deaths per 100,000 people under 65 in our primary service area

86

Performance Measure Example: Governor’s Committee on Employment of People with Disabilities

Vision = To have the highest competitive employment rate of people with disabilities in the U.S.Possible Performance Measure = ?

87

Performance Measure Example: Governor’s Committee on Employment of People with Disabilities

Vision = To have the highest competitive employment rate of people with disabilities in the U.S.

Possible Performance Measure =

Percent of people with disabilities in competitive employment

88

Performance Measure Example: Prison Industries

Philosophy = All staff will act in a fashion that presents to the inmates an ethical and moral model, that were they to emulate, they would never return to prison.


89

Performance Measure Example: Prison Industries

Philosophy = All staff will act in a fashion that presents to the inmates an ethical and moral model, that were they to emulate, they would never return to prison.

Possible Performance Measure = Percent of inmates who worked for Prison Industries who return to prison

90

Class Exercise

Using the agency performance measures list on page 15 of your handout, decide what type of performance measure each item is (outcome, output, etc.)

91

Types of Measures



92

Class Exercise

Using the “Performance Measure or Not?” list on p. 16 of your worksheet packet, find an item on the list that is not a performance measure and write a measure for it.

93

Table Exercise

Using the performance measures for an agency at your table, decide what types of measures they are (output, outcome, etc.) and complete the Performance Measures Checklist on pg. 17 of your handout

94

Types of Measures



95

Tool for Measures

To help draft your agency’s measures, consider the following (see page 4 of handout):



to ______________ (whom)


96

Tool for Measures

Where in the statement is an output?

Where is an outcome?

Where is a caseload?



to ______________ (whom)


97

Class Exercise

Using the tool to create measures for:

Output Efficiency Outcome Caseload

98

Talk About Percent

Percent of [universe] that is [something in particular]

Percent of class attendees who are female

Percent of McDonald’s balls that are green

99

Talk About PercentPercent of [universe] that is [something in particular]

Numerator is [particular]

Denominator is [universe]Number of green balls

Total number of balls

100

Talk About PercentNumerator and denominator must each be entered separately in NEBS (Budget system)

Number of green balls Total number of ballsUse real #s, not 70 and 100

101

Class Exercise (Optional)

Using the performance measures on pp. 18-22, decide what types of measures they are (output, outcome, etc.)Improve at least one of the measures, and/or create an outcome measure for the agency.

102

Table Exercise

Decide what types of measures your agency has (output, outcome, etc.)Improve at least one of the measuresCreate an outcome measure for your agency.

103

In Summary, Measures Should Be…

UnderstandableOutcome orientedUsefulValid, Verifiable and Accurate AvailableComparable/Consistent

104

Performance Measures: Collect the Data

Establish a baseline

05

10152025303540

time

Average Wait Time Per Customer

FY 12

FY 13

Linear (FY 13)

105

Calculate the Measure

Are you getting there?

Compare to the objective, goal, etc that sparked the measure

What are your customers saying?

Record your results.

106

Don’t Forget the Auditors…

Keep a record of:

Where you get the data. Be specific: titles of reports, which

files, who maintains them.

How to calculate the measures. Specific calculation

procedures/formulas.

107

Don’t Forget the Auditors…

Have supervisory program & fiscal staff review the calculations.

Keep the underlying data for 3 years.

Send the values and definitions to Archives.

See handout for records retention.

108

Now that you wrote a plan…

Plan how to implement it.Assign a minder.Review: how are we doing?

109

Back to the Beginning

The cycle repeats!Modify strategies if your data shows what you’re doing isn’t working.Make needed changes to programs/processes.

110

Continuous ProcessStrategic Planning

Develop Measures

Collect DataAnalyze Data

Revise Mgmt Processes & S/P

US Dept. of the Treasury, Financial Management Service

111

Contact Information

Maud Naroll, Chief PlannerDepartment of AdministrationResearch, Planning, Grants 775-684-0223 [email protected]

the performance measure problem…

Documents

strategic plans

strategic planninganswers

strategic planningwhatan

clean onlyclean sewers

agency personnel

miles of sewers

lot of sewers

epstein fass associates