step 7: comparison of alternatives 7a.compare coas using alternative selection criteria to identify...
TRANSCRIPT
Step 7: Comparison of Alternatives
7a. Compare COAs using alternative selection criteria to identify the preferred COA.
7b. If there is a bill associated with the recommended COA, identify the billpayer.
7c. Identify the positive and negative impacts of the second- and third-order effects. What must be done to manage the negative impacts?
7d. Determine the robustness of the conclusions. If anything changes – assumptions, costs, benefits, etc. – would the recommendation change?
7e. Identify the high-risk aspects of the recommended COA and define appropriate risk mitigation measures.
1 CBA Training Slides
Heart of Comparative Analysis
The essence of the CBA process is in comparing two or more courses of action in order to identify the preferred alternative.
As a general rule, the preferred alternative is the alternative that provides the greatest amount of benefit in relation to its cost (Best Value!).
Value = Benefit – Costs
2UNCLASSIFIED CBA Training Slides
CBA is Value Analysis
Aid for Completing Step 7a
3UNCLASSIFIED CBA Training Slides
Costs Benefits Selection Criteria
EqualUnequal Alternative that provides greatest benefits for
given cost
Equal Subjective reasoning and a fortiori analysis
UnequalUnequal
Alternatives ranked in order (based on benefit/cost ratio, net present value, or other relevant criterion)
Equal Least costly alternative
Best Practices
For most comparative analyses delta costs are more informative
Present the single (or two) most important benefit(s) to compare against cost
Find the “knee in the curve” or optimal value solution
4UNCLASSIFIED CBA Training Slides
Common Mistakes
Assuming away the problem Assuming away cost “Over-averaging” for sake of simplification
Show examples of each (or at least last one)
5UNCLASSIFIED CBA Training Slides
Simple Case
Cost ($M) Benefit
Status Quo 30 Good
Added capability 45 Very Good
Decreased capability 55 Excellent
6UNCLASSIFIED CBA Training Slides
Which COA is the best value?
Data Analysis
7UNCLASSIFIED CBA Training Slides
Costs Benefits
30 150
45 225
55 275
• Data in and of itself is of little use.• Analysis takes data and puts it into a format that enables
us to make better decisions.
Optimization
8UNCLASSIFIED CBA Training Slides
Costs Benefits30 15045 22555 275
Costs Benefits30 15040 16045 22550 26555 275
Fairly linear relationship
Decreasing bang for buck
Increasing bang for buck
Which COA is the best value?
Optimization example
Find the “knee in the curve” Simple linear example
use simplex Teach solver
Efficient frontier? – or complex example
9UNCLASSIFIED CBA Training Slides
Normalization of Value
Facilitates easy value comparison of COAs Allows comparison of cost today to cost tomorrow Allows comparison of present and future benefits Allows comparison of costs and benefits
Appropriate method must be chosen from many choices
Costs and benefits may have to be recalculated based upon chosen method
10UNCLASSIFIED CBA Training Slides
(Net) Present Value
The difference between the present value of cash inflows and the present value of cash outflows.
Used to analyze the profitability of an investment Costs (outflows) and benefits (inflows)are in
dollars Value presented as a single number Only works if costs and benefits are monetary
11UNCLASSIFIED CBA Training Slides
PV Merits
Pros Most fundamental
analysis Very easy to compare
results Incorporates discount
rate which can be represent risk
Cons Limited use in the
Army Benefits are purely
monetary Discount rate
application can be difficult
12UNCLASSIFIED CBA Training Slides
Example
Which costs more? Present costs in base year. Present costs in present value or NPV.
13UNCLASSIFIED CBA Training Slides
Current/Then year $MM
2000 2001 2002 2003 2004 2005 2006 TOTALCOA 1 (85) (21) (21) (22) (23) (23) (24) (218)COA 2 (150) (10) (11) (11) (11) (12) (12) (217)
Inflation 3%Discount Rate 5%
PV Example
Which costs more?
14UNCLASSIFIED CBA Training Slides
Current/Then year $MM
2000 2001 2002 2003 2004 2005 2006 TOTALCOA 1 (85) (21) (21) (22) (23) (23) (24) (218)COA 2 (150) (10) (11) (11) (11) (12) (12) (217)
2000 2001 2002 2003 2004 2005 2006 TOTALCOA 1 (85) (20) (20) (20) (20) (20) (20) (205)COA 2 (150) (10) (10) (10) (10) (10) (10) (210)
Constant/Base year $MM
2000 2001 2002 2003 2004 2005 2006 TOTALCOA 1 (85) (20) (19) (19) (19) (18) (18) (197)COA 2 (150) (10) (10) (9) (9) (9) (9) (206)
PV $MM discounted at 5% rate
PV Example
Relative costs Compare annual savings Discounts at cost of capital
15UNCLASSIFIED CBA Training Slides
Current/Then year $MM
NPV $MM discounted at 5% rate
2000 2001 2002 2003 2004 2005 2006 TOTALCOA 1 (85) 0 0 0 0 0 0 (85)COA 2 (150) 10 11 11 11 12 12 (83)
2000 2001 2002 2003 2004 2005 2006 TOTALCOA 1 (85) 0 0 0 0 0 0 (85)COA 2 (150) 10 10 9 9 9 9 (94)
Benefit/Cost Ratio
Easy to compare different alternatives Total benefit obtained per unit of cost Projects with greater BCRs are usually
given priority over those with smaller BCRs
Alternatives that have a BCR greater than one (1) are considered viable
16UNCLASSIFIED CBA Training Slides
BCR Examples
Simplified example with dollars Operational example
Utility/dollar Dollars/kill
17UNCLASSIFIED CBA Training Slides
Decision Matrix
Tool compares benefits and costs to produce a single value score
Army staff officers taught to include in decision briefs
18UNCLASSIFIED CBA Training Slides
Decision Matrix Merits
Pros Easy to use Normalizes costs and
benefits Familiar tool
Cons Easy to introduce
error Highly judgmental Discourages accurate
identification of value Easy to argue against
results Scoring is difficult
19UNCLASSIFIED CBA Training Slides
Extraneous Data
Cost Drivers capture the primary costs of a proposal
Given a list of benefits, is there a primary consideration?
Focusing on a long list can distract decision makers from true impacts
20UNCLASSIFIED CBA Training Slides
Example
Benefits: Increased transaction rate Better working area Faster email processing Less administrative burden
How do you present the value?
21UNCLASSIFIED CBA Training Slides
InvestmentCosts
Transaction Rates Email Speed
Administrative Load
Status Quo 30 150Decent ~ 30 min
delay/week -
COA 1 45 225Better ~10 min
delaySave 300 hours/year
COA 2 55 275Much Better ~5
min delaySave 1000 hours/year
“Cost Free” Analysis
Alternatives present savings against the status quo or baseline case
Or the analysis costs are non-monetary
22UNCLASSIFIED CBA Training Slides
“Free” Example
Problem: After a Congressional inquiry, G-3/5/7 proposes formation of a new organization to track and coordinate MRAP related issues similar to DAMO-AV, meets 10 new requirements, I to X. Status Quo: No new office, requirements I - V currently met by
AMC and TACOM. COA 1: Fill organization with 23 Military and Civilian personnel
by eliminating current AMC MRAP office of 30. COA 2: Fill organization with 23 Military and Civilian TACOM
personnel.
23UNCLASSIFIED CBA Training Slides
“Free” Example
Costs Benefits
Status Quo – Baseline
COA 1 •With loss of office, AMC will not be able to meet requirements X &Y.•Lease costs near Pentagon•Transfer of personnel & equipment•Locality differences
•Reqts VI-X.•Cost of 7 personnel•Huntsville facility savings
COA 2 •TACOM will no longer be able to meet requirements A, B & C.•Lease costs near Pentagon•Transfer of personnel & equipment•Locality differences
•Reqts VI-X.•TACOM facility savings
24UNCLASSIFIED CBA Training Slides
Double Counting of Criteria
25UNCLASSIFIED CBA Training Slides
COA1-SQ COA2-CIV COA3-MIL
Cost Costs $0.0 6.0 $5.0 2.0 $5.0 2.0
BenefitARFORGEN NA 2.0 GOOD 3.0 BAD 1.0
New mission benefit 0 1.0 +10% 3.0 +10% 3.0
9.0 8.0 6.0
COA1-SQ COA2-CIV COA3-MIL
Cost Costs $0.0 6.0 $5.0 4.0 $10.0 2.0
BenefitARFORGEN NA 2.0 GOOD 3.0 BAD 1.0
New mission benefit 0 1.0 +10% 3.0 +10% 3.0
9.0 10.0 6.0
Impact of using personnel is counted both as a cost and a contra-benefit
Step 7b: Billpayers
Billpayers are the funding sources that have been identified to cover the cost of the recommended COA. In most cases, the individual or team developing the CBA won’t have
the authority to identify billpayers. This requires collaboration with the organization’s resource manager and prioritizer.
Savings can be used as a billpayer, but cost avoidances cannot. Savings: A cost reduction that enables a manager to move funds from one
function to another Cost avoidance: A cost reduction that does not enable a manager to move
funds.
26UNCLASSIFIED CBA Training Slides
Step 7c: 2nd- and 3rd-Order Effects
Second- and third-order effects are the “ripple effect” of the recommended COA: “The recommended COA will solve our problem, but it will also create an additional factor we will have to deal with.”
The cost of many CBAs is purely a 2nd or 3rd order effect (e.g. policy change).
The scope or domain often limit the # of next order effects.
27UNCLASSIFIED CBA Training Slides
Risk Assessment
Risk assessment describes risks that can impact the achievement of stated benefits or the cost of solving the business problem. For each risk, assess the likelihood of occurrence and develop a mitigation strategy.
Ways to incorporate risk into value calculation:
Sensitivity analysis
Discount factors
28UNCLASSIFIED CBA Training Slides
Sensitivity Analysis
Sensitivity analysis identifies the impact on the recommendation should any element of the analysis change.
Sensitivity analysis is a good means to address risk impacts
29UNCLASSIFIED CBA Training Slides
Example
Variations in assumptions that change cost and benefits
@risk or crystal ball run Change costs change benefits Change decmat weights
30UNCLASSIFIED CBA Training Slides
Questions for Steps 7d and 7e
How sensitive is the recommendation to possible changes in costs, benefits, assumptions, etc? If the recommendation is highly sensitive to changes, has more in-depth analysis been done?
Which elements of the CBA require sensitivity analysis? Test only those elements for which there is considerable uncertainty or
risk. Can include any element: Assumptions, constraints, costs, benefits,
weighting of selection criteria, etc.
Address sensitivity from either or both perspectives: What is the impact of a change of such and such a magnitude? How large a change can occur before the recommendation changes?
Have all reasonably likely risks and their impacts been identified? Are the recommended mitigation approaches adequate? Are they affordable?
31UNCLASSIFIED CBA Training Slides
Step 8: Report Results and Recommendations
Preferred format for documenting the CBA is a narrative. This will probably be accompanied by a decision briefing.
Results and recommendations summarize the analysis and make conclusive statements about the comparisons of alternatives.
The results address how the alternatives were ranked using the criteria developed in Step 6. Following a clear statement of the conclusions, there should be a firm recommendation regarding the preferred alternative.
All data and other information used in Steps 1-8 must be adequately documented. Supporting information should be identified so decision makers and analysts can understand how Steps 1-8 were developed.
32
Questions for the reviewer:Does the package contain all key elements, accompanied by supporting documentation?Does the recommended COA address the problem, and is it consistent with the assumptions and constraints?Does the analysis explain how the recommended COA is best at satisfying the selection criteria?
CBA Training Slides
Practice Step 8
We will discuss Step 8 by reviewing a notional CBA decision briefing for the APS practical exercise
Note: The CBA Guide, available at the Cost & Performance Portal, includes a narrative report for the APS example.
33UNCLASSIFIED CBA Training Slides
Decision package must present a strong value proposition. That is, it must clearly show that the benefits of the recommended
COA more than justify the costs and risks.