10.15.2013 prj & port mgmt sftdev - investment analyzer - projection and simulations
DESCRIPTION
10.15.2013 Prj & Port Mgmt SftDev - Investment Analyzer - Projection and SimulationsTRANSCRIPT
© 2013 IBM Corporation
AAO-CO202: IT and Engineering Portfolio & Strategy Management
Jim DensmoreExecutive IT Specialist
IT Architect and Technical Sales SpecialistRational Brand, SWG
23 October 2013
© 2013 IBM Corporation
IBM
2
2013 – Project and Portfolio Management in Systems and Software Development
Rational Focal Point Investment Analysis ComponentProjection and Simulations 15 October 2013
Jim Densmore Murray Cantor, Ph.D.Sr. Certified IT Specialist Distinguished EngineerIBM Rational Software IBM Rational [email protected] [email protected]
© 2013 IBM Corporation
© 2013 IBM Corporation
IBM
3
Objectives
Motivate and understand an ROI-based method for you and your clients to align the development organization (IT, Engineering, etc.) with the business
See how IBM Rational provides a mechanism for clients to do this more easily
– Process
– Tooling
– Use cases
© 2013 IBM Corporation
IBM
4
Agenda• Business Challenges
• Aligning IT or Engineering with the Business
• An ROI based Solution
• Focal Point and its Investment Analysis Component
• Use Cases for Investment Analysis
© 2013 IBM Corporation
IBM
5
Agenda• Business Challenges
• Aligning IT or Engineering with the Business
• An ROI based Solution
• Focal Point and its Investment Analysis Component
• Use Cases for Investment Analysis
© 2013 IBM Corporation
IBM
6
Business ChallengesZero-growth budgets
0.0%
2.0%
4.0%
6.0%
8.0%
10.0%
12.0%
14.0%
16.0%
18.0%
200819980.0%
2.0%
4.0%
6.0%
8.0%
10.0%
12.0%
14.0%
16.0%
18.0%
20072006200520042003200120001999
Trend Line
15.0% 15.9%
9.7% 10.1%
2002
1.3%
0.0%1.6% 2.5% 2.7%
3.2%3.0%
Source: Gartner
Executive Programs CIO SurveyIncreases in CIO Budgets Over Previous Year, 1998-2009 (Worldwide)
2009
0.16%
2011 Demand for IT Services“The economic downturn forced deep cuts in IT budgets. Now, as CIO’s plan for the recovery, they are facing unprecedented demand for IT services from the business. At the same time, organizations are still keeping spending tightly under control.”
CIOUpdate.com August 3, 2010http://www.cioupdate.com/budgets/article.php/3896646/How-to-Get-the-Budget-You-Need-in-2011.htm
With Less!
Do More!
6
© 2013 IBM Corporation
IBM
7
Often, ongoing operations & maintenance consume > 2/3 of budgets This may leave too little for innovation
Business Response Must (re!)balance resource allocations to support business innovation
7
Original graphic created by David Puzas, WW Marketing Executive for IBM Enterprise Services, 2007
IT S
pe
nd
IT S
pe
nd
2/3
1/3
© 2013 IBM Corporation
IBM
8
Agenda• Business Challenges
• Aligning IT or Engineering with the Business
• An ROI based Solution
• Focal Point and its Investment Analysis Component
• Use Cases for Investment Analysis
© 2013 IBM Corporation
IBM
9
Alignment of IT (or Engineering) & Business Value is Crucial
A disciplined, strategic translation is key to this alignment• Translation management and maintenance?
• What is their common denominator?
What challenges must we meet?• Business values evolve
• The mapping to capabilities changes
• Capabilities are always in flux
We can’t fund both We can’t fund both projects. Which one projects. Which one
do we keep?do we keep?
© 2013 IBM Corporation
IBM
10
The conventional wisdom for estimating the worth of an incomplete development effort (project) provides little insight
Fails to acknowledge work already done
Provides no opportunity for ongoing value management
0 6 Ship date
Conventional wisdom: no value until shippedV
alu
e
? ? ? ? What scale do What scale do youyou use? use?
How does one avoid gaming?How does one avoid gaming?
Have you ever killed a project? How?Have you ever killed a project? How?
?
© 2013 IBM Corporation
IBM
11
Agenda• Business Challenges
• Aligning IT or Engineering with the Business
• An ROI based Solution
• Focal Point and its Investment Analysis Component
• Use Cases for Investment Analysis
© 2013 IBM Corporation
IBM
12
Inception Elaboration Construction Transition/Maintenance
New capability on existing platform Maintenance and
small change requests
New platform
Medium VarianceHigh Variance Low Variance
Bringing IBM’s math department to bear
Time
Var
ian
ce in
co
st, s
ched
ule
…
• Different activities different analytics & productivity measures
• Improve odds of delivery and predictability across the lifecycle
Industry spend
ROI
© 2013 IBM Corporation
IBM
13
This led to the approach: what might someone pay for it?
Imagine selling your incomplete development program
–Buyer would spend now/today
–Obtains the option to invest in its completion
–Once completed, receives hoped-for benefits
–So … buy the option to spend to receive uncertain benefits
• This is like a call option with an uncertain strike price!
• No wonder ROI can be difficult to measure
Reason about a fair price using incomplete market reasoning: Expected cost to complete over time,
Expected valuevalue received over time, The estimated risk (uncertainty) over time
© 2013 IBM Corporation
IBM
14
For what might Airbus sell their For what might Airbus sell their A350 XWB program?A350 XWB program?
– In testing with reportedly good resultsIn testing with reportedly good results
– Maiden flight completed Maiden flight completed (June 2013)(June 2013)
– Many orders at varying levels of firmnessMany orders at varying levels of firmness• 682 a/c ordered by 33 global customers682 a/c ordered by 33 global customers• As of July 2013 (wikipedia)As of July 2013 (wikipedia)
Certainly not zero!Certainly not zero!
Example
? ? ? ? How much value do you see in your How much value do you see in your
portfolio of incomplete projects? portfolio of incomplete projects? Does the value grow over time?Does the value grow over time?
(What’s wrong with this image? (What’s wrong with this image? Why?)Why?)
?
© 2013 IBM Corporation
IBM
15
Agenda• Business Challenges
• Aligning IT or Engineering with the Business
• An ROI based Solution
• Focal Point and its Investment Analysis Component
• Use Cases for Investment Analysis
© 2013 IBM Corporation
IBM
16
Rational Focal PointHardwiring the linkage between strategy and execution
• The Rational Team Concert integration allows users to prioritize and manage project scope and rollup project status
• The System Architect integration connects the Enterprise Architecture perspective to the portfolio management perspective in Focal Point
• FP’s Investment Analysis component assists users with financial modeling and business case assessment
• Users can take advantage of advanced resource management allowing skill-based supply and demand tracking and balancing
• Configuration templates included in the product helps users get up and running quickly
© 2013 IBM Corporation
IBM
17
The best common scale of value is “Money” Investment Analysis provides financial measures for everyday use
© 2013 IBM Corporation
IBM
18
IA presents streams with quarter-by-quarter values
Explicit values
(others are interpolatedothers are interpolated)
© 2013 IBM Corporation
IBM
19
Each input assumption is found using simple input estimates
A triangular distribution is used– Commonly used in business decision making
– Values are simple to understand
– (Remember, the area of the triangle is 1 by definition)
0
E, the most likely or expected monetary value
L, the lowest monetary value you believe could occur (no chance of a lower value)
H, the highest monetary value you believe could occur (no chance of a higher value)
L E H
© 2013 IBM Corporation
IBM
20
Vignette: How long will it take?
You’re a development manager
You assign your developer a task
“How long will it take?” He hems & haws …
“Oh … about 20 weeks!”
The developer returns to his cube
His buddy prairie dogs & says, “Hey. Did you get the job?” “Yeah.” “How long will it take?” “Oh, about 8 weeks!”
… What happened?
© 2013 IBM Corporation
IBM
21
Vignette: How long will it take?
You’re a development manager
You assign your developer a task
“How long will it take?” He hems & haws …
“Oh … about 20 weeks!”
The developer returns to his cube
His buddy prairie dogs & says, “Hey. Did you get the job?” “Yeah.” “How long will it take?” “Oh, about 8 weeks!”
… What happened?
0 8 12 20
Conclusion: ask for all three numbers!
The three-number (triangular distribution)
answer is both more honest as well as
more proactive; it changes the game
© 2013 IBM Corporation
IBM
22
IA represents uncertainty with 3 values for each quarter (period)
© 2013 IBM Corporation
IBM
23
You assign a discount rate on each (stakeholder) page
Discount rate determination is an entire, important subject on its own:
It should never be zero (the default, by the way)
Should costs and benefits have the same or different rate?
Too high and long-term investment is discouraged
Too low? Might be worth doing, but mañana is soon enough!
? ? ? ? How do you handle discount rates?How do you handle discount rates? ?
© 2013 IBM Corporation
IBM
24
A view (page) can be developed for each Stakeholder
Revenue Devel. Costs
Maint. Costs Combined
© 2013 IBM Corporation
IBM
25
Monte Carlo analysis permits the needed arithmetic operations on random variables
Run thousands of simulations of the investment value (the NPV)
– Each time a value is picked from each triangularly distributed random variable
– Each value chosen is based on the likelihood of that value occurring, according to the distribution of the associated random variable
For each simulation,the investment valueis calculated
Finally, we build up the histogram of investment values toobtain its distribution
© 2013 IBM Corporation
IBM
26
A practical demonstration of the value of risk managementment
Revenue Devel. Costs
Maint. Costs Combined
What is the effect of narrowing cost variance?
© 2013 IBM Corporation
IBM
27
This shows Monte Carlo analysis demonstrating empirically the value of narrowing uncertainties in cost
© 2013 IBM Corporation
IBM
28
Thus, our model for estimating NPV and ROI has some useful properties
Investment value varies continuously in time
Investment value improves when we invest:
–In improving likelihood of delivery (reducing uncertainty in costs)
– In improving the range of value, e.g. building reuse into solutions (that is, increasing the upside variance of benefits)
0 6 Ship date
Conventional WisdomV
alu
e
Investment
Value
© 2013 IBM Corporation
IBM
29
Bubble chart: standard way to represent a project portfolio
1
2
3
Risk
Str
ateg
ic V
alu
e
0 5
4
5
xx Doesn’t show that incomplete projects have valueCreates a discontinuity in thinkingRarely accounts for timing and total costs of ownership
Traditionally, value & risk are based mostly on perceptions, opinions
Project 62
10
8
4
3
9
5
6
1
7
$$
Radius = Cost
“better”
“worse”
© 2013 IBM Corporation
IBM
30
This Investment Value Model puts monetary number$ on value and risk
xInvestment Value = Mean
Standard Deviation IV = MeanNormalized Risk = (Scaled) Standard Deviation??
Project 1
Project 2 Project 3
Project 4
Project 5
Project 7
Project 8
Project 9
Project 6
Project 10
2
10
8
4
3
9
5
6
1
7
-1000
-500
0
500
1000
1500
2000
Normalized Risk
Val
ue
0 1
Project 1
Project 2 Project 3
Project 4
Project 5
Project 7
Project 8
Project 9
Project 6
Project 10
2
10
8
4
3
9
5
6
1
7
-1000
-500
0
500
1000
1500
2000
Normalized Risk
Val
ue
0 1
$
© 2013 IBM Corporation
IBM
31
Agenda• Business Challenges
• Aligning IT or Engineering with the Business
• An ROI based Solution
• Focal Point and its Investment Analysis Component
• Use Cases for Investment Analysis
© 2013 IBM Corporation
IBM
32
These two key questions support value-based decision making
Are we creating value?Are we creating value? Is this program worth continuing?Is this program worth continuing?
Program onset: T0 Program delivery: Td
Monitoring Is program healthy? Intervene?Cut losses?
Investment Is program still needed?Should we adjust content?Should we continue to invest?
Likely value at delivery, Likely value at delivery, & likely ROI at delivery& likely ROI at delivery
Current value, Current value, ROI to dateROI to date
Management Decisions Supported:
Today: T1
? ? ? ? How do you compare routine and How do you compare routine and
innovative efforts to each other?innovative efforts to each other?
How do you manage project risk?How do you manage project risk?
How do you motivate architectural How do you motivate architectural robustness and reuse?robustness and reuse?
?
© 2013 IBM Corporation
IBM
33
The Model permits more objective management of the portfolio in the usual resource-constrained environment
$1k
$2k
$3k
Normalized Risk
$ V
alu
e
0 1
$4k
$5k
2
10
8
4
3
9
5
6
1
7
Project 62
10
8
4
3
9
5
6
1
7
Legend
Keep
Questionable
Cut?
© 2013 IBM Corporation
IBM
34
The model can help choose among build scenarios
Scenario A – Cobbled together, quick to field, minimum investment
Scenario B – More reusable, extensible, lower O&M costs
$1k
$2k
$3k
Normalized Risk
$ V
alu
e
0 1
$4k
$5k
A
B
© 2013 IBM Corporation
IBM
35
Another use is to track improvement in value, and reduction of risk, throughout the project lifecycle
T1 is project onset; T2 and T3 are later times in the lifecycle
Movement from lower right to upper left shows that the investment (development) is delivering value
$1k
$2k
$3k
Normalized Risk
$ V
alu
e
0 1
$4k
$5k
T1
T2
T3
© 2013 IBM Corporation
IBM
36
IA is a key part of Delivering on these Five Keys to Success
1. Understand the value of investments and tradeoffs on content rather than becoming “bogged down” in project or resource detail
2. Build a stronger connection between customers, Enterprise Architecture and portfolio management disciplines to enable a broader viewpoint for decisions
3. Leverage lifecycle management of products, services or IT applications with customer, competitive and capability viewpoints that show how projects deliver against the lifecycle
4. Gain insight into how delivery practices impact project success, to inform process and skill improvements and organizational maturity
5. Provide teams with a collaborative platform that allows software delivery projects to become more automated, transparent and predictable across all disciplines
Its difficult to link an IT project portfolio to the realization of business benefits
Its difficult to link an IT project portfolio to the realization of business benefits
Organizations do not have the ability to reconcile both project delivery and
architectural perspectives
Organizations do not have the ability to reconcile both project delivery and
architectural perspectives
Organizations want to manage lifecycles, not just projects
Organizations want to manage lifecycles, not just projects
Project managers report on the status of a project, but this does not help improve
delivery capabilities within the organization
Project managers report on the status of a project, but this does not help improve
delivery capabilities within the organization
Project management tools focus on the needs of the project manager helping them manage schedules, costs and resources but creates
overhead for practitioners
Project management tools focus on the needs of the project manager helping them manage schedules, costs and resources but creates
overhead for practitioners
Blockers Five keys to success
© 2013 IBM Corporation
IBM
37
AAO-CO202
© 2013 IBM Corporation
IBM
38
© Copyright IBM Corporation 2013. All rights reserved. The information contained in these materials is provided for informational purposes only, and is provided AS IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, these materials. Nothing contained in these materials is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software. References in these materials to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates. Product release dates and/or capabilities referenced in these materials may change at any time at IBM’s sole discretion based on market opportunities or other factors, and are not intended to be a commitment to future product or feature availability in any way. IBM, the IBM logo, Rational, the Rational logo, Telelogic, the Telelogic logo, and other IBM products and services are trademarks of the International Business Machines Corporation, in the United States, other countries or both. Other company, product, or service names may be trademarks or service marks of others.
www.ibm.com/software/rational
AAO-CO202
© 2013 IBM Corporation
IBM
39
For more interaction …
Join and collaborate on the TLE Community.
– TLE Community: http://w3.ibm.com/connections/communities/service/html/communityview?communityUuid=eb5b200a-0890-4a21-b06d-dc4141c34f20
TLE events are being scheduled continuously, so check the website frequently. – TLE Website: http://tle.atlanta.ibm.com/home.html
© 2013 IBM Corporation
IBM
40
Alignment is Difficult
Line of Business ExecutiveLine of Business Executive
Project Manager or Team Lead
Project Manager or Team Lead
Senior ManagerSenior Manager
Profit, Internal Rate of ReturnProfit, Internal Rate of Return
Project deliverables, cost and schedule
Project deliverables, cost and schedule
Delivery of business value through the optimal use of resources
Delivery of business value through the optimal use of resources
• Concerns flow down the organization while measures (and data) flow up • We need tools to plan, track, and deliver on our commitments at every level
Concerns
Mea
sure
s
measures
commits to
measures
measures
commits to
commits to
© 2013 IBM Corporation
IBM
41
Motivators and Blockers
Line of Business Executive
Line of Business Executive
Project Manager or Team Lead
Project Manager or Team Lead
Senior ManagerSenior Manager
Need more effective support for strategic decision
making
Need better metrics & analytics for
continuous process improvement
Must connect better with practitioners,
enhance production
Linking IT project portfolio to business benefit realization
Linking IT project portfolio to business benefit realization
Reconciling project delivery & architectural perspectives
Reconciling project delivery & architectural perspectives
Want to manage lifecycles, not just projects
Want to manage lifecycles, not just projects
PMs’ status reports don’t improve delivery capabilitiesPMs’ status reports don’t
improve delivery capabilities
PM tools focus on PM needs but increase
overhead for practitioners
PM tools focus on PM needs but increase
overhead for practitioners
© 2013 IBM Corporation
IBM
42
Monte Carlo Simulation (Wikipedia1 has a good article at http://en.wikipedia.org/wiki/Monte_Carlo_method)
Monte Carlo methods (or Monte Carlo experiments) are a class of computational algorithms that rely on repeated random sampling to compute their results. Monte Carlo methods are often used in simulating physical and mathematical systems. These methods are most suited to calculation by a computer and tend to be used when it is infeasible to compute an exact result with a deterministic algorithm. This method is also used to complement the theoretical derivations.
Monte Carlo methods are especially useful for simulating systems with many coupled degrees of freedom, such as fluids, disordered materials, strongly coupled solids, and cellular structures (see cellular Potts model). They are used to model phenomena with significant uncertainty in inputs, such as the calculation of risk in business. They are widely used in mathematics, for example to evaluate multidimensional definite integrals with complicated boundary conditions. When Monte Carlo simulations have been applied in space exploration and oil exploration, their predictions of failures, cost overruns and schedule overruns are routinely better than human intuition or alternative "soft" methods.
The Monte Carlo method was coined in the 1940s by John von Neumann, Stanislaw Ulam and Nicholas Metropolis, while they were working on nuclear weapon projects in the Los Alamos National Laboratory. It was named in homage to Monte Carlo casino, a famous casino, where Ulam's uncle would often gamble away his money.
Introduction and example: Monte Carlo method applied to approximating the value of π
Monte Carlo methods vary, but tend to follow a particular pattern:– Define a domain of possible inputs. – Generate inputs randomly from a probability distribution over the domain. – Perform a deterministic computation on the inputs. – Aggregate the results.
For example, given that a circle inscribed in a square and the square itself have a ratio of areas that is π/4, the value of π can be approximated using a Monte Carlo method:
– Draw a square on the ground, then inscribe a circle within it. – Uniformly scatter some objects of uniform size (grains of rice or sand) over the square. – Count the number of objects inside the circle and the total number of objects. – The ratio of the two counts is an estimate of the ratio of the two areas, which is π/4. Multiply the result by 4 to estimate π.
In this procedure the domain of inputs is the square that circumscribes our circle. We generate random inputs by scattering grains over the square then perform a computation on each input (test whether it falls within the circle). Finally, we aggregate the results to obtain our final result, the approximation of π.
To get an accurate approximation for π this procedure should have two other common properties of Monte Carlo methods. First, the inputs should truly be random. If grains are purposefully dropped into only the center of the circle, they will not be uniformly distributed, and so our approximation will be poor. Second, there should be a large number of inputs. The approximation will generally be poor if only a few grains are randomly dropped into the whole square. On average, the approximation improves as more grains are dropped.
Monte Carlo methods (or Monte Carlo experiments) are a class of computational algorithms that rely on repeated random sampling to compute their results. Monte Carlo methods are often used in simulating physical and mathematical systems. These methods are most suited to calculation by a computer and tend to be used when it is infeasible to compute an exact result with a deterministic algorithm. This method is also used to complement the theoretical derivations.
Monte Carlo methods are especially useful for simulating systems with many coupled degrees of freedom, such as fluids, disordered materials, strongly coupled solids, and cellular structures (see cellular Potts model). They are used to model phenomena with significant uncertainty in inputs, such as the calculation of risk in business. They are widely used in mathematics, for example to evaluate multidimensional definite integrals with complicated boundary conditions. When Monte Carlo simulations have been applied in space exploration and oil exploration, their predictions of failures, cost overruns and schedule overruns are routinely better than human intuition or alternative "soft" methods.
The Monte Carlo method was coined in the 1940s by John von Neumann, Stanislaw Ulam and Nicholas Metropolis, while they were working on nuclear weapon projects in the Los Alamos National Laboratory. It was named in homage to Monte Carlo casino, a famous casino, where Ulam's uncle would often gamble away his money.
Introduction and example: Monte Carlo method applied to approximating the value of π
Monte Carlo methods vary, but tend to follow a particular pattern:– Define a domain of possible inputs. – Generate inputs randomly from a probability distribution over the domain. – Perform a deterministic computation on the inputs. – Aggregate the results.
For example, given that a circle inscribed in a square and the square itself have a ratio of areas that is π/4, the value of π can be approximated using a Monte Carlo method:
– Draw a square on the ground, then inscribe a circle within it. – Uniformly scatter some objects of uniform size (grains of rice or sand) over the square. – Count the number of objects inside the circle and the total number of objects. – The ratio of the two counts is an estimate of the ratio of the two areas, which is π/4. Multiply the result by 4 to estimate π.
In this procedure the domain of inputs is the square that circumscribes our circle. We generate random inputs by scattering grains over the square then perform a computation on each input (test whether it falls within the circle). Finally, we aggregate the results to obtain our final result, the approximation of π.
To get an accurate approximation for π this procedure should have two other common properties of Monte Carlo methods. First, the inputs should truly be random. If grains are purposefully dropped into only the center of the circle, they will not be uniformly distributed, and so our approximation will be poor. Second, there should be a large number of inputs. The approximation will generally be poor if only a few grains are randomly dropped into the whole square. On average, the approximation improves as more grains are dropped.