january 2014 – public release 1.0

January 2014 – Public Release 1.0

Performance Benchmarking for Effectively Managed Water Utilities

(Project 4313)

User Guide for the Self-Assessment Tool

Updated 01-07-14

Overview

The Self-Assessment Tool includes Practice Areas for each of the 10 EUM Attributes as well as key metrics to measure performance for each of those practices.

The Tool enables utilities to select and weigh their relevant attributes and practices, and then assess those practices for both current and target performance.

This guide explains the steps necessary to use the tool and leverage its capabilities.

For additional information on Effective Utility Management, please visit http://watereum.org/

http://watereum.org/

Updated 01-07-14

Select Attributes

Select Practice Areas

Select Performance

Measures

Create Custom

Assessment

Assess each Performance

Measure

Verify Assessment

Review Attribute

Summaries

Return to Setup

Delete Performance

Measures

View Summary Results

Model Roadmap

Each of the following steps will be addressed in the guide:

Updated 01-07-14

Opening the Tool

1. When the Tool is opened, a simple disclosure “splash screen” is presented. Exit the application or start the EUM Tool by clicking the associated button.

Updated 01-07-14

Attributes

1. Weigh each Attribute in terms of relative importance to the utility compared with the other selected Attributes.

If a weight of 0 is selected, the Attribute will not be included in the assessment; otherwise scales of 1-10 or 1-100 are examples of weighting options that will work.

2. After weighting, click this button to navigate to the next screen.

Select Attributes


Select Performance

Measures

Create Custom

Assessment


Measure

Updated 01-07-14

Practice Areas

1. Only Attributes that have been weighed appear on this screen. Next, weigh the relative importance of each Practice Area that should be included in the assessment.

As with the Attributes, if a weight of 0 is selected, the Practice Area will not be included in the assessment; otherwise a scales of 1-10 or 1-100 are examples of weighting options that will work.

2. After weighting, click this button to navigate to the next screen.

Select Attributes


Select Performance

Measures

Create Custom

Assessment


Measure

Updated 01-07-14

Performance Measures

1. Enter “Y” or “N” to select Performance Measures that should be included in the assessment.

Select Attributes


Select Performance

Measures

Create Custom

Assessment


Measure

2. After choosing the Performance Measures, click this button to create the customized self-assessment.

Updated 01-07-14

Select Attributes


Select Performance

Measures

Create Custom

Assessment


Measure

2. The Tool also creates summary sheets for each of the selected Attributes containing the current and target scores along with the weighted totals.

3. The next step is to select these links or navigate through the Performance Measure tabs to assess each Performance Measure.

1. The Tool creates a scoring matrix for each of the Performance Measures under each Attribute. The worksheet names are coded using the Attribute #, Practice Area #, and Performance Measure #.

Customized Menu

Updated 01-07-14

Select Attributes


Select Performance

Measures

Create Custom

Assessment


Measure

1. The column and row headings can be edited by double-clicking the appropriate cell if you’d like to modify the pre-defined criteria descriptions.

4. Clicking the squares within the matrix will turn on/off “Current,” “Target,” or “Current & Target.”

5. Select current performance before selecting a target. Clicking activated squares will reset them for re-scoring.

6. After completing all of the performance measure assessments, click on

2. Find the intersection between the correct level of performance with degree of implementation for your utility. Click to enter your current performance and repeat the process to enter the target.

Perform Assessment

3. In defining the performance measures, some rows and columns were intentionally left blank. Please use caution to avoid selecting the blank rows/columns.

Updated 01-07-14

Verify Assessment

Review Attribute

SummariesReturn to

Setup

Delete Performance

Measures


1. Clicking this button will test to see if there are missing data that must be entered before continuing.

2. Use these links to navigate to specific Attribute(s) for the next step.

Verify Assessment

Updated 01-07-14

Verify Assessment

Review Attribute

SummariesReturn to

Setup

Delete Performance

Measures


1. The first potential error is that a Performance Measure has not yet been assessed. This is resolved by navigating to the appropriate Performance Measure(s) and recording the current and target performance.

2. The second potential error is that a Practice Area has been weighted, but no Performance Measures have been selected. This is resolved by either setting the Practice Area weight to zero or entering “Y” in one or more Performance Measures related to that Practice Area.

3. The third potential scoring error is that an Attribute has been weighted, but no Practice Areas have been selected. This is resolved by either setting the Attribute weight to zero, or entering a numerical weight for one or more Practices Areas related to that Attribute, and then entering “Y” in one or more Performance measures related to each selected Practice Area.

Potential Assessment Errors

Updated 01-07-14

Review Attributes

The individual Performance Measures are listed, with links to their sheets. If necessary, click the links to revise assessments.

Verify Assessment

Review Attribute

SummariesReturn to

Setup

Delete Performance

Measures


Current and target performance scores are listed based on the matrix selections.

Practice Area scores (arithmetic mean of Performance Measure scores) are listed in the table and illustrated in the graph.

Updated 01-07-14

The following two slides illustrate the process that can be used to add/remove assessment elements and to delete Performance Measures, respectively.

Removing assessment elements takes them out of the assessment summaries but does not delete the underlying Performance Measure sheets and scoring from the tool;

Deleting Performance Measures deletes the underlying Performance Measure sheets and scoring from the tool.

Removing vs. Deleting

Updated 01-07-14

Return to SetupVerify Assessment

Review Attribut

e Summa

ries

Return to

Setup

Delete Performance Measur

es

View Summa

ry Results

If you determine that it is appropriate to add or remove elements from what is shown in the assessment, you can add or remove Attributes, Practice Areas, and Performance Measures, even after scoring, by selecting the element and making appropriate adjustments.

Updated 01-07-14

Delete Performance MeasuresVerify Assessment

Review Attribut

e Summa

ries

Return to

Setup


es

View Summa

ry Results

1. If it is determined that certain Performance Measures are no longer needed, click this button to remove them.

2. This pop-up window will appear with each of the Performance Measures listed. Click the check boxes for the Performance Measures you’d like to remove and then click OK.

3. This process does not affect the settings on the setup menu; therefore , in order to prevent the deleted Performance Measure(s) from reappearing, they should be switched from “Y” to “N” in the setup menu.

Updated 01-07-14

Review Results

After the assessment has been completed, the summary results for each Attribute (based on the Practice Area weights) are listed and illustrated on the graph. The total weighted utility score is also provided, based on the specific Attribute weights.

Verify Assessment

Review Attribut

e Summa

ries

Return to

Setup


es

View Summa

ry Results

List of Performance Measures

Updated 01-07-14

1. Product Quality

1. Comply with Regulatory and Reliability Requirements1. Degree of Success in Achieving Target Drinking Water Compliance Rate (percent)2. Degree of Success in Achieving Target Wastewater Treatment Effectiveness Rate (percent)3. Degree to Which Near Compliance Misses are Minimized (water)4. Degree to Which Near Compliance Misses are Minimized (wastewater)5. Meet National and Other Applicable Objectives for Residuals Quantity and Quality6. Extent to Which Supplemental Local Goals are Addressed

2. Address Customer Needs1. Achievement of Target Drinking Water Flow and Pressure (percent)2. Achievement of Target Fire Suppression Flow and Pressure (percent)3. Extent to Which Service Interruptions are Reduced to Target Levels (percent)4. Extent to Which Sewer Backups are Reduced to Target Levels (percent)5. Extent to Which Sanitary Sewer Overflows are Reduced to Target Levels6. Extent to Which Combined Sewer Overflows are Reduced to Target Levels7. Extent to Which Safe Drinking Water Act Secondary Standards are met

3. Address Public Health and Ecological Needs1. Achievement of Water Reuse Targets (percent)2. Achievement of Beneficial Biosolids Usage Targets (percent)

Updated 01-07-14

2. Customer Satisfaction

1. Minimize Customer Complaints1. Extent to Which Customer Service Complaint Rates Fall within Target Levels2. Extent to Which Technical Quality Complaint Rates Fall within Target Levels

2. Achieve Target Level of Customer Service Delivery1. Degree of Field Call Responsiveness (water)2. Degree of Field Call Responsiveness (wastewater)3. Degree to Which Error-driven Billing Adjustments Are Minimized4. Degree to Which Service Start/Stop Responsiveness Falls Within Target Levels5. Extent to Which Customer Issues are Resolved by Call Center/Customer Service at the time of

the First Call 6. Degree to Which Customer Service Responsiveness Falls Within Target Levels7. Degree of Abandoned Calls

3. Receive Positive Customer Perceptions1. Degree of Positive Customer Feedback Received2. Perception of Effectiveness at Dispute Management and Resolution

4. Efficiently Deliver Customer Service1. Efficiency of Customer Service (Ratio of Cost/O&M Spending)

Updated 01-07-14

3. Employee and Leadership Development

1. Recruit Appropriate Talent1. Degree of Success in Implementing a Recruitment Strategy2. Extent to Which Job Descriptions are Implemented3. Adequacy of Interview and Selection Processes

2. Retain Existing Talent1. Degree of Implementation of Retention Management Plans2. Degree of Implementation of Onboarding Programs3. Degree of Implementation of Job Satisfaction Programs4. Degree of Success in Employee Engagement

3. Address Succession Planning Needs1. Adequacy of Workforce Forecasting 2. Adequacy of Succession Plan

4. Strengthen Core Competencies1. Degree of Success in Implementing Performance Management Systems2. Degree of Implementation of Learning Programs3. Level of Management Training Achieved

Updated 01-07-14

4. Operational Optimization

1. Provide for Ongoing Operational Improvements1. Track Record in Providing for Ongoing Operational Improvements2. Extent to which Operational and Technology Improvements Are Deployed3. Extent to Which Automated Regulatory Monitoring & Reporting Systems are Implemented

(Water)4. Extent to Which Automated Regulatory Monitoring & Reporting Systems are Implemented

(Wastewater)2. Minimize Resource Use and Losses from Day to Day Operations

1. Degree of Implementation of Resource Optimization Plans2. Degree of Energy Optimization3. Degree of Labor Optimization4. Degree of Chemicals Optimization5. Degree of Residuals Optimization6. Degree of Maintenance Optimization

Updated 01-07-14

5. Financial Viability

1. Develop Sound Financial Plan1. Degree to Which Financial Planning Efforts Support Strong Bond Ratings2. Degree to Which Rate Increase Level is Supported by Planning Studies3. Appropriateness of Rate Planning Horizon4. Appropriateness of Balance of Capital Spending Between Debt and Equity Expenditures

2. Provide Financial Integrity1. Financial Policy Integrity2. Financial Procedural Integrity

3. Achieve Budget Management Effectiveness1. Appropriateness of Annual Revenue to Expenditure Ratio (operating)2. Appropriateness of Annual Revenue to Expenditure Ratio (capital)3. Appropriateness of Annual Revenue to Expenditure Ratio (Overall)4. Degree to Which Long-term Life-cycle Accounting Employs Risk-Based Considerations5. Adequacy of Operating Reserves

Updated 01-07-14

6. Infrastructure Stability

1. Develop and Implement an Asset Management Program1. Degree of Implementation of an Asset Management (AM) Framework2. Degree of Implementation of Levels of Service3. Degree of Development of an Asset Management Plan

2. Maintain Knowledge of Assets and Costs1. Level of Asset Inventory2. Level of Asset Condition Information3. Level of Asset Performance Information4. Availability of Cost of Asset Ownership Information

3. Incorporate Risk-Based Analysis into Decisions1. Degree to Which Risk of Failure Analysis is Incorporated in Decision-Making2. Extent to Which Critical Assets are Identified 3. Extent to Which Risk Mitigation Options are Identified and Evaluated as Part of Decision-Making

Updated 01-07-14

7. Operational Resiliency

1. Incorporate Risk Assessments into Decision-making1. Extent to Which Vulnerability Assessments (VAs) are Conducted and Recommendations

Implemented2. Implement Risk Mitigation

1. Emergency Response and Recovery Planning2. Business Continuity Planning / Continuity of Operations Planning3. Crisis Communications4. Business Risk

3. Sustain Employee Resiliency1. Achievement in Employee Injury Time Reduction2. Health and Safety Training3. Health and Safety Plan

Updated 01-07-14

8. Community Sustainability

1. Utility Organization1. Leadership2. Sustainability Plan3. Management System4. Sustainability Reporting

2. Infrastructure Project Sustainability1. Project Planning, Design, and

Engineering2. Infrastructure Procurement3. General Procurement

3. Natural Environment1. Water Management2. Water Conservation3. Stormwater4. Energy5. Waste Reduction6. Ecosystem Services7. Pollution Prevention8. Climate Change Adaptation

4. Economic Strength1. Economic Competitiveness2. Local Sourcing3. Local Payroll

5. Social Equity1. Use of Strategic Financial Planning to

Minimize Overall Rate Impacts2. Appropriate Use of Affordability Programs

to Protect Specially Impacted Groups3. Degree of Success in Keeping Customer

Bills Within Affordable Levels4. Community Support

Updated 01-07-14

9. Water Resource Adequacy

1. Achieve Water Supply Adequacy1. Adequacy of Long-Term Water Supply2. Adequacy of the Ratio of Short-term Supply to Short-term Demand (12-mo rolling average)3. Degree of Implementation of a Basin Supply Plan4. Degree of Implementation of Drought Management Plan5. Degree of Implementation of a Source Water Protection Plan

2. Optimize Reduction of Non-Revenue Water1. Degree of Success in Achieving Target Levels of Annual Non-revenue Water (NRW)2. Degree of Success in Achieving Target % of Real Water Loss

3. Implement Water Conservation1. Degree of Success in Achieving Target Per Capita Use

4. Achieve Water Supply Reliability1. Degree of Implementation of Demand Management/Demand Reduction Plans2. Frequency of Full Satisfaction of Demand3. Consideration of Supply/Demand Uncertainty4. Degree of Success in Managing Raw Water Turbidity to Target Levels

Updated 01-07-14

10.Stakeholder Understanding and Support

1. Stakeholder Identification1. Degree of Success in Stakeholder Identification and Analysis

2. Stakeholder Engagement Plan1. Degree of Success in Developing and Executing a Stakeholder Engagement Plan

3. Oversight Body Engagement Strategy1. Success in Gaining Oversight Body Understanding

4. Media Interaction Program1. Degree of Success in Interacting with Media (Traditional and Emerging) to Achieve Accurate

Coverage2. Degree of Success in Establishing Positive Coverage by the Media (Traditional and Emerging)

5. Stakeholder Support Performance Measurement System1. Level of Stakeholder Support Received for Critical Strategic and Operational Directions

january 2014 – public release 1.0

Documents