Program Performance Accountability& Measurable Results
Kevin Keaney, ChiefPesticide Worker Safety Programs
U. S. EPA2005
OPP
Outline
Performance Measurement: Context, Value, Types
Accountability Drivers & Problems
GPRA & PART
Elements of a Good Measures Program
Where Accountability Fits StrategicallyProgram Success is built from critical components.
CLEAR GOALSPublic Health & Ecological
Protection Outcomes
GOOD PLANNINGStrategic Objectives
EFFECTIVE IMPLEMENTATIONMultiple Levels
ADEQUATE RESOURCESAcquisition & Allocation
MEANINGFUL ACCOUNTABILITYMeasures & Tracking
OPP
Value of Performance/Results Data
MULTIPLE USES Guiding Program Management: Decisions;
Priorities; Resources
Performance Accountability Tracking program success and progress
Communication
General Public Interested Stakeholders Decision Makers (Congress, Admin.)
Meaningful Accountability = Good Management = Good Government
5
OPP
Types of Performance Measures
Performance measures vary depending on level of organization & need.
Strategic GoalsBroader/Long-TermBroader/Long-Term
Specific/Short-TermSpecific/Short-Term
Mid-RangeObjectives
AnnualCommitments
IndividualPerformance Stds
OPP
Accountability Drivers / Mechanisms
Statutes
EPA 5 Year Strategic Plan
3 Year NPM Commitments
Budget Formulation
Annual Work Plans
Budget Allocation
Tracking Systems/Reporting
Audits
Performance Standards
OPP
Problems w/ Current Systems
Components:…are complex,…are created at different times,…by different people,…for different purposes,…little thought about connections or alignment among the parts.
Inevitably creates:…conflicts,
…confusion, …unnecessary burdens, &
…mixed messages.
Doesn’t work: …no accountability …no consequences.
OPP
Accountability Overload
FIFRA!
GPRA!
REGIONALINITIATIVES!
OECA!LAWSUITS!
STRATEGICPLAN!
OMB!PART!
BUDGET!
SPECIALPROJECTS!
MEDIACOVERAGE!
FLEXIBILITY!FOCUS! POLITICAL
PRESSURE!
OPP
Accountability Drivers/Mechanisms
Statutes
EPA 5 Year Strategic Plan
3 Year NPM Commitments
Budget Formulation
Annual Work Plans
Budget Allocation
Tracking Systems/Reporting
Audits
Performance Standards
OPP
Accountability Drivers/Mechanisms
Statutes
EPA 5 Year Strategic Plan
3 Year NPM Commitments
Budget Formulation
Annual Work Plans
Budget Allocation
Tracking Systems/Reporting
Audits
Performance Standards
OPP
GPRA
GPRA = Government Performance and Results Act of 1993
GPRA is a statute aimed at increasing accountability by requiring measurement of program results & linking those results to resource allocation.
Provides a statutory framework for strategic planning and reporting that requires federal agencies to:
Establish accountability measures to assess the outcomes of each program activity, and
Prepare performance plans that establish annual performance goals.
OPP
GPRA Auditing
Program Assessment Rating Tool (PART)
Office of Management & Budget assumed the role of evaluating progress against GPRA.
OMB is committed to audit all federally funded programs against GPRA requirements on a 5 year cycle. Size of individual programs receiving detailed audit varies.
This is the third year. Several Pesticide programs have been audited.
OPP
GPRA Audits
OMB
Fire is invented
OPP
GPRA Auditing
Program Assessment Rating Tool (PART)A matrix of questions with a numeric value & requiring
succinct answers.Principle focus has been on the existence of Outcome
Measures satisfying OMB.OMB has not been bound by pre-existing measures
(EPA’s Strategic Plan; NPM Guidance).Moving target of changing requirements.Short turn-around times.Average initial scores of programs have been very low
(40 - 60 out of 100) and many classified as “Results Not Demonstrated”.
“re-PARTing” is the norm.Results are published each year with the federal budget.Programs not demonstrating results for successive
years are subject to budget cuts or elimination.
OPP
PART Budget Consequences
EPA
OMB
OPP
Criteria for Good Measures
VALUEHIGH
OutcomesClear
Wide use
LOW
BeansComplex
Narrow use
MeaningfulUnderstandable
Useful
BURDENLIGHT
FewSimple
LowUser-friendly
HEAVY
ManyHardHigh
Technical
# of MeasuresDifficulty of Collection
Data System CostsExpertise Required
Performance Measures Quadrant Analysis
High Value
Low Value
LightBurden
HeavyBurden
OUTCOMENIRVANA
OUTPUTHELL
1. High Value/Light Burden 2. High Value/Heavy Burden
4. Low Value/Heavy Burden3. Low Value/Light Burden
Measures Comparison Matrix
VALUE BURDEN
Meaningful Understandable UsefulPossible Measures
Measures Criteria
OPP
Process for Measures Improvement
Specific Problem Formulation Baseline data collection: Current measures; current
data systems, etc. Scope of project? Strategic measures only or a
universal results ethic (measures for all projects). Participants? OPP; OECA; Regions; States/Tribes;
industry; NGOs; etc. Outputs? Measures for Strategic Plan/NPM/Annual
Workplans/PART; Public Reports; Change risk assessment/management practices; Change or consolidate databases.
Timeline/Deadlines? 6 months (PART); 2 years (next 5 Year Strategic Plan).
Process Structure
Senior Level Steering CommitteeSet Direction; Manage Process; Resolve conflicts / issues
Program Specific WorkgroupsGather Data; Coordinate; Develop Measures + Data Systems
Regis. WPC&TESARegis. Review
SAI
Coordinate w/ Partners & Stakeholders
Regions USDAOW,ORD,
OECAIndustry
States,Tribes
NGOs
Databases & Information SystemsEvaluate existing sources; Opportunities for cost savings
OPP
Take Away Message
All government funded activities will be held to higher standards of performance accountability to justify budgets.
Cooperative work is required to establish meaningful accountability measures and performance targets.
Well documented progress toward quantifiable public health and environmental protection goals will help ensure continued program operation and funding.
OPP
Program Performance Accountability& Measurable Results
QUESTIONS ?