dif 8901 no. 1 cots-based systems top 10 lists victor r. basili, barry boeham addresser: hao ding...
TRANSCRIPT
DIF 8901 No. 1
COTS-Based Systems Top 10 Lists
Victor R. Basili, Barry Boeham
Addresser: Hao Ding
NTNU/IDI, March 5th, 2003
DIF 8901 No. 2
What is this paper for? Top 10 in COTS-Based Systems (CBS) Conclusion
Outline
DIF 8901 No. 3
What is this paper for? Intention:
starting point for the software community: developing solid empirical methods for coping with COTS-Based data.
Characteristics of CBS: The buyer has no access to the source code; The vendor controls its development; Nontrivial installed base (>one customer, >a few
copies)
DIF 8901 No. 4
Criteria for making the list: Each empirical result has significant current
and future impact on software dependability, timeliness, and cost;
Diagnostic value with respect to cost-effective best practices;
Reasonable generality across applications domains, market sectors, and product sizes.
What is this paper for? (con’d)
DIF 8901 No. 5
What is this paper for? (con’d)
Maintenance
DisplacementImplementation
ContagionInitiation
Visibility
Recommended Adoption Time Frame
Type A Type B Type CTime
BandwagonEffect
Slope ofEnlightenment
Plateau ofProductivity
TechnologyTrigger
Trough of Disillusionment
Peak of Inflated
Expectations
Software Adoption Curve
DIF 8901 No. 6
Top 10 in COTS-Based Systems (CBS) Hypotheses rather than results Software challenges for enhancing our empirical
understanding of CBS 10 hypothesis:
DIF 8901 No. 7
Hyperthesis One
These are the same criteria we used for More than 99 percent of all executing computer instructions come from COTS products. Each instruction passed a market test for value.
DIF 8901 No. 8
Hyperthesis Two
More than half the features in large COTS software products go unused.
DIF 8901 No. 9
Hypothesis Three
The average COTS software product undergoes a new release every eight to nine months, with active vendor support for only its latest three releases.
DIF 8901 No. 10
Hypothesis Four
CBS development and post-deployment efforts can scale as high as the square of the number of independently developed COTS products targeted for integration.
That is, integrating n COTS products involves potentially n(n - 1)/2 interfaces.
DIF 8901 No. 11
Hypothesis Five
CBS post-deployment costs exceed CBS development costs
Example: A cost-constrained developer on a 27-month project might be tempted to present a maintainer from a different organization with about-to-become-unsupported COTS products.
DIF 8901 No. 12
Hypothesis Six
Although glue-code development usually accounts for less than half the total CBS software development effort, the effort per line of glue code averages about three times the effort per line of developed-applications code.
DIF 8901 No. 13
Hypothesis Seven
Non-development costs, such as licensing fees, are significant, and projects must plan for and optimize them
DIF 8901 No. 14
Hypothesis Eight
CBS assessment and tailoring efforts vary significantly by COTS product classes –operating system, database management system, user interface, device driver, and so forth.
DIF 8901 No. 15
Personal capability and experience remain the dominant factors influencing CBS-development productivity.
Hypothesis Nine
DIF 8901 No. 16
CBS is currently a high-risk activity, with effort and schedule overruns exceeding non-CBS software overruns. Yet many systems have used COTS successfully for cost reduction and early delivery.
The Standish Group’s CHAOS report (http://www.scs.carleton.ca/~beau/PM/Standish-Report.html )
Hypothesis Ten
DIF 8901 No. 17
Intention of the list Reprenting preliminary hypotheses for validity-checking
CBS project decisions against small empirical data samples with explained variablility sources
Starting point for the software community
For more general information: http://www.sei.cmu.edu/cbs/ http://www.cebase.org/cots/
<END>
Conclusion
DIF 8901 No. 18
Populating Software RepositoriesIncentives and Domain-Specific Software
Jeffrey S. Poulin
Addresser: Hao Ding
NTNU/IDI, March 5th, 2003
DIF 8901 No. 19
What is this paper for? Intention:
This article presents IBM’s experiences with a corporate RSL reuse incentive programs, and summarizes the results of an enterprise-wide initiative to develop reusable software for the RSL.
DIF 8901 No. 20
RSL RSL - Reusable Software Library The place to deposit software for use Do not guarantee success Depending on factors like availability of quality
and useful software Domain-specific considerations most often
determine the usefulness of software and should therefore influence the population of an RSL
DIF 8901 No. 21
IBM’s RTSC IBM formed the Reuse Technology Support Centre
(RTSC) to promote formal reuse throughout the corporation
Promoting and forwarding “formal” reuse throughout the corporation
RTSC worked with designated representative from development sites
RTSC sponsored enhancement to an existing library system so that it could support and control worldwide distribution of assets
DIF 8901 No. 22
Reuse library considerations 1980’s : reuse programs focused on RSL Quite often ”the reuse program become the
library” Resources are spent on library related activities,
not on the reuse methods and technology. Large RSLs suffer from many drawbacks that
reduce their effectiveness.
DIF 8901 No. 23
The Reuse Library Progression Phase 1
very few parts Phase 2
many parts of low or poor quality Phase 3
many parts of little or no use
DIF 8901 No. 24
RSL overhead
Overhead caused by the formality and rigor needed to manage large quantities of datae.g.. Extensive classification scheme used to describe software components
DIF 8901 No. 25
A Corporate Reuse Library The IBM strategic RSL provides a
central repository for sharing, managing, and reusing software-related products to company cites worldwide
A distributed system of shared libraries The library is owned by the ”local” organization
and access can be granted to RSL users from other sites
Based on a detailed classification scheme to support locating of relevant software
Software in the library to be reused is copied to the local machine
DIF 8901 No. 26
Component management in the RSL Intention:
To assist library managers: Component management functions:
Library access control Component user registration Version control Problem reporting Library shadowing Standard enforcement
DIF 8901 No. 27
Reuse Incentive Programs RTSC in 1991 faced a corporate RSL in the first
stage of the three phases Feedback indicated a need for a wider selection
of quality and useful software RTSC did not sponsor a centralized incentive
program but encouraged and offered guidance to sites that wanted to start or have their own
DIF 8901 No. 28
The supplier-consumer relationship
The supplier those who write and contribute quality
reusable software and related information for others
The reusers those who extract from the RSL and
incorporates the software in their products rather than develop it again (the consumer)
DIF 8901 No. 29
Recognizing suppliers When determining the proper recognition the
program should address: Participation
allocating points to first-time contributors helps motivating developers and generates interest
Popularity Provide more recognition to contributors of very popular
components encourages contributors to look for high-reuse opportunities in their projects
Value A large, robust and generalized component provides more
economic benefit to the organization than small, specific components
DIF 8901 No. 30
SIRBO SIRBO - Source Instructions Reused By Others Recognizes part suppliers The size (line of codes) for each component the
individual contributes by the number of products or organizations actually using it.
Helps ensure that suppliers do not receive recognition for unused software
Rewards suppliers who identify widely needed functions
Rewards contribution of large parts
DIF 8901 No. 31
Recognizing Reusers Reuse results from a business analysis showing possible
financial savings to the organization Forms the basis for individual recognition Recognizing reusers also helps create a demand for
reusable components that leads to financial benefits. RSI – Reused Source Instructions
It provides an excellent metric for rewarding reusers. How to calculate the RSI for a reuser?
Counting the number of code lines in each reused component Credit is received first time component appears in a
product
DIF 8901 No. 32
Recognizing teams A team incentive can reward a number of group
activities: reward developer teams who achieve
a pre-specified goal for the level of reuse receive an unforeseen exceptionally high level of reuse
reward ”parts centre” based on SIRBO
Reward any team with a high number of individuals participating in reuse activities
DIF 8901 No. 33
A corporate incentive program Local reuse programs at IBM started to develop
and mature, but the overall quantity of parts available for general corporate use seemed to stabilize
RTSC sponsored the development of reusable software ”The Parts Stimulation Program”
Moving from phase 1 without the pitfalls of phase 2 and 3
DIF 8901 No. 34
The Parts Stimulation Program Strategy getting development organizations to realize that
the projects would be more valuable if they made portions of the software reusable
the program funded the dev. organizations to study their projects and the business issues related to reuse
the Parts Stimulation Program funded the front-end expenses of conducting the reuse study
performing a domain analysis developing the specification for the reusable parts completing a reuse business case detailing who else
would benefit and the expected return of investments
DIF 8901 No. 35
Incentive Program Results RTSC decided to fund alternative projects that might
benefit the corporation reuse standard documents development tools study of an area of future development
Parts Stimulation Program Summary in RSTC
DIF 8901 No. 36
What makes software usable Programmers cannot reuse a software unless it is
useful Categories for software
DIF 8901 No. 37
Domain independent software single function or set of variables higher-level abstractions may include numerous
interrelated data and functions that act together to perform a specific task
high quality rarely contributes more than 15 –20 % to the total
product making it easy to reuse domain-independent
reusable components is an easy, straight-forward and low cost way to get started in reuse
DIF 8901 No. 38
Domain specific software Findings in various projects show high level of
domain specific reuse The best results will come from a domain that:
has limited scope programmers understand has matured to a level of relative stability requires a lot of custom development based on a set of
core functions Creating a domain library can be bottom-up or
top-down
DIF 8901 No. 39
Conclusion Examination on the implementation of a
corporate RSL Discussion on the local and large-scale reuse
incentive programs that aim to populate the RSL Discussion on the relationship between domain.
<END>
DIF 8901 No. 40
A Comparison of Approaches to Investment Analysis
John Favaro
Addresser: Hao Ding
NTNU/IDI, March 5th, 2003
DIF 8901 No. 41
What is this paper for? Background:
Although significant progress has been made in the areas of reuse metrics and cost estimation, much work to date in reuse investment analysis has not always reflected accepted mainstream financial analysis practices.
Intention:Comparing several approaches that have been described in the reuse literature, points out known problems and indicates remedies.
DIF 8901 No. 42
Software reuse economics Three kinds of
activities: Reuse metrics
the measurement of reuse-related characteristics of software
Cost estimation the estimation of
cost and benefits associated with reusable software development
Reuse investment analysis the evaluation of
investment decisions
DIF 8901 No. 43
The investment analysis Is in the domain of the financial corporate analyst Different perspective from the software engineer
reuse is only one alternative for investment is concerned with the best way to allocate capital and
human resources There is always an alternative to reuse
investment equivalent investment in the capital market that
provides some yearly rate of return reuse-oriented investment should be compared with this
Capital investments are analysed with respect to periods of time, and the same approach should be used for reuse investment analysis
DIF 8901 No. 44
Cost estimation and cash flow Cost estimation = cash flow analysis Estimation of cost/benefit Candidate reuse projects has potential cash flows
positive (e.g.. savings by avoiding work) negative (e.g.. work to generalize a component)
Techniques for quantifying economic benefits are emerging.
Cash flows are forecast over a suitable time horizon
Time period must be neutral estimate, not a desired characteristic
Cash flow analysis is an activity prior to investment analysis
DIF 8901 No. 45
Comparison of approaches
Desirable characteristics depend as much as possible only on forecast
cash flow have a quantifiable acceptance rule to guide
the investment decision suitable for comparing and ranking candidate
projects be able to deal with arbitrarily large or small
projects be able to handle projects of arbitrarily long or
short duration
DIF 8901 No. 46
Net present value PV = Ct / (1 + kt)t
PV = Present Value Ct : future cash flow in period t kt : discount rate in period t
NPV = C0 + PV NPV = Net Present Value add the initial investment as a negative value IF NPV>0 THEN can be accepted
DIF 8901 No. 47
NPV characteristics Acceptance rule is based on the value of NPV Permits realistic comparison with alternative
capital investment possibilities Does not depend on arbitrary factors (e.g..
managers instincts) Values are additive, can be ranked, and are
sensitive to scale Large and small alternative can be compared and
combined
DIF 8901 No. 48
Payback
The time or number of uses required to recover the cost of an investment
The payoff threshold value: N0 = E / (1-b)
N0 = number of times a component must be used before its cost is recovered
E = relative cost of developing a component for reuse
b = relative cost of integrating the component
DIF 8901 No. 49
Payback characteristics Acceptance is based on cost recovery within a
cut-off date Intuitively appealing but problematic
cut-off date is arbitrarily and subjective payback is not sensitive to patterns of cash flow problem of scale: the true value (long term) is not taken
into account choice of cut-off period affects whether short or long
lived projects are accepted, tends to penalize forward looking reuse programs
Ad-hoc approach useful for communicating the result of an investment analysis
DIF 8901 No. 50
Average return on book value Software as a capital asset An amortization schedule for the investment for
reusable work products is agreed upon (deducted as appropriate from future cash flows from those work products)
The book rate of return (of an investment) is calculated by:
dividing the avg. profits from predicted cash flows by the avg. net book value of the investment
DIF 8901 No. 51
Avg. return on book value characteristics Acceptance rule is based on the book rate of
return meets some target set by the analyst e.g.. companies current book rate of return of the industry as a whole
The approach is entirely insensitive to a variable cash flow pattern
The calculation is depending on the choice of amortization schedule
Choice of target book rate is arbitrary and subjective
Not satisfactory approach
DIF 8901 No. 52
Internal rate of return
A way of defining the rate of return of a long lived asset
Internal Rate Return (IRR) is related to NPV IRR is the discount rate which makes NPV
equal to zero
C0 = Ct / (1 + IRR)t
Ct : future cash flow in period t
DIF 8901 No. 53
IRR characteristics Acceptance rule:
accept a project if its IRR is greater than the opportunity costs of the project
Can be used to produce results equivalent to NPV Proper use of IRR is more difficult Can be undermined by certain patterns of cash
flow in a project Exhibits problems with respect to the scale of
projects
DIF 8901 No. 54
Profitability Index
Examines the ratio of benefits to costs Conceptually closest to NPV Numerous variations
ROI (return of investment) Q (Quality of investment) CDCF (Cumulative discounted cash flow)
Values are note additive
DIF 8901 No. 55
Conclusion Net Present Value (NPV) Payback Average Return on Book Value Internal Rate of Return (IRR) Profitability Index (PI)
<END>