evaluating results - scottsdale...2017/03/16  · “what gets measured, gets managed.” peter...

EVALUATING RESULTS

How can we manage

performance in local

government?

“What gets measured,

gets managed.”

Peter Drucker

Performance management is

an organization-wide effort to

improve results by integrating

objective evidence with decision-

making processes

“What matters is not

finding the perfect

indicator, but settling

upon a consistent and

intelligent method of

assessing your output

results and then tracking

your trajectory with rigor.”

Jim Collins

DEFINE

RESULTS

ACHIEVE

RESULTS

ASSESS

RESULTS

IMPROVE

RESULTS

BY CONDUCTING

ACTIVITIES

BY DELIVERING

SERVICES

THAT BENEFIT

CUSTOMERS

WE ACHIEVE

RESULTS

THAT USE

RESOURCES

WITH DRIVERS AND

TRUCKS

BY DRIVING TO EACH

HOUSE TWICE A WEEK

BY COLLECTING REFUSE

AND RECYCLING

FROM EVERY

HOUSEHOLD

WE ENCOURAGE A

CLEAN, SUSTAINABLE

ENVIRONMENT

BY RECRUITING

APPLICANTS

THAT ARE QUALIFIED

DRIVERS

WITH ANALYSTS,

COMPUTERS AND

WEBSITES

WITH DRIVERS AND

TRUCKS

BY DRIVING TO EACH

HOUSE TWICE A WEEK

BY COLLECTING REFUSE

AND RECYCLING

FROM EVERY

HOUSEHOLD

WE ENCOURAGE A

CLEAN, SUSTAINABLE

ENVIRONMENT

InputAmount of resources used (or available) to provide services

OutputAmount of work produced or

services delivered

OutcomeThe desired end result that demonstrates the impact of

the services delivered

EfficiencyAmount of work done per amount of resources used

QualityAmount of quality work done per amount of resources used

EffectivenessAmount of achieved results, or the level of quality relative to

the amount of work done

Cost-EffectivenessAmount of outcome achieved per amount of resources used

Types of performance measures

InputEquipment Operators

OutputTons collected per month

OutcomeRefuse is collected in a reliable

and clean manner

EfficiencyTons collected per month

per operator

QualityTons of complaint-free collection per operator

EffectivenessTons collected per month

without complaints of missed or messy collection

Cost-EffectivenessCost to provide residential

refuse collection services per operator

Solid Waste Example

Program Evaluation

What can you find out about the problem?

What results did you receive from your analysis?

How might you test your hypothesis?

Why do you think the problem is occurring?

What problem are you trying to solve?PURPOSE

RESEARCH

HYPOTHESIS

EXPERIMENT

ANALYSIS

What is the best way to report your conclusions?CONCLUSION

SCI

ENTI

FIC M

ETH

OD

FO

R E

VA

LUA

TIO

N

What is the problem? What is happening?

How do we improve the condition?

How do we address the cause?

Why should anyone care? What is the impact?

Why did the condition happen?

How do we know this is a problem? What should be?CRITERIA

CONDITION

CAUSE

EFFECT

RECOMMENDATION

FIVE ELEMENTS OF A FINDING

GAO Yellow Book: http://www.gao.gov/assets/590/587281.pdf

The problem (or opportunity) is x.

I recommend doing y.

The benefit of this is z.

I also considered doing a or b.

They fall short of y,

because of d and e.So Smart But…, Weiner, 2007

core values for analysts:

empathy

humility

curiosity

“People… operate with beliefs and biases.

To the extent you can eliminate both and replace them

with data, you gain a clear advantage.”

Michael Lewis | Moneyball: The Art of Winning an Unfair Game

Is police officer pay fair?

History of Market and Merit Increases

Merit

Market

3 %Fix

Scottsdale Typical compared to 5-city step average base salary

Completed Years of Service

Because of step programs and increases there is a gap between the Valley average and current Scottsdale salaries

Completed Years of Service

Scottsdale is higher in these years

Scottsdale is lowerin these years

Scottsdale

67 (21%)

Gilbert

45 (14%)

Phoenix

66 (20%)

Mesa

45 (14%)

Chandler

20 (6%)

Tempe

9 (3%)

Glendale

11 (3%)

All other: 18 (6%)

Where do Scottsdale

Police Officers Live?

(by zip code)

Do we need another

engine company downtown?

First Response Unit - Station 602 Area

FY 2011/121.9 companies

Station 6026,161 Calls

FY 2012/131.1 companies

Station 6026,784 Calls

23% OTHERS601 13%

603 7%

PHX 4%

TEMPE .3%

7% OTHERS601 3%

603 3%

PHX 1%

TEMPE .2%

93% 77%

10% more calls

How do we increase

voluntary donations?

Improve your reporting

“effective information visualization (is) premised on:

simplicity (complex notions simplified to save time for reader);

transparency(visual honesty and responsibility in sourcing);

creativity(design that is memorable and understandable);

sociability(easily shared and improved)”

http://www.edelman.com/p/6-a-m/visualizing-information/@Sambrook

Edward R. Tufte

The Visual Display of Quantitative Information, Edward R. Tufte, 2001

@EdwardTufte

Remove Chartjunk

4.3

2.5

3.5

4.5

2.4

4.4

1.8

2.8

0

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

1 2 3 4 5 6 7 8

Chart Title

Series1

“interior decoration of graphics

generates a lot of ink that does not

tell the viewer anything new.”

Increase Data-Ink

High - 4.5

Low - 1.8

$2.8 million

1 2 3 4 5 6 7 8

“data graphics should draw the viewer’s

attention to the sense and substance of

the data, not to something else.”

Perfection is finally

attained not when there

is no longer anything to

add…

Antoine de Saint Exupéry

but when there is no

longer anything to take

away ...

M. Brent Stockwell | Asst. City Manager480-312-7288 | Bstockwell@ScottsdaleAZ.gov

go to ScottsdaleAZ.gov search “performance”