1 results of reuse survey jared fortune, usc ricardo valerdi, mit gan wang, bae cosysmo workshop @...

1

University of Southern California Center for Systems and Software Engineering

Results of Reuse Survey

Jared Fortune, USC

Ricardo Valerdi, MIT

Gan Wang, BAE

COSYSMO Workshop @ COCOMO Forum 2008

Los Angeles, CA

2


Outline

• Research Background

• State of the Practice Survey

• Results

• Implications for COSYSMO 2.0

3


COSYSMO Reuse Development Timeline

COSYSMO 2.02005 2006 2007 2008

COSYSMO 1.0 Published [1]

Reuse Identified as a Critical Improvement [2]

Extensions for COSYSMO to Represent Reuse Published [3]

Preliminary Categories of Reuse Identified [4]

COSYSMO-R Developed At Lockheed Martin [5]

Reuse Definitions Identified [6]

BAE Pilot Test Shows Promising Results [8]

Other Reuse Considerations Identified [10]

Survey Results on State of the Practice [11]

Reuse Continuum Identified;“Bottoms-Up”Approach Proposed [7]

“Bottoms-Up”Results from BAE Presented [9]

2009

[1] Valerdi. COSYSMO. Ph.D. Dissertation,2005.[2] Valerdi. COSYSMO Workshop. USC ARR, 2006.[3] Valerdi, Gaffney, Roedler, Rieff. COSYSMO Extensions. COCOMO Forum, 2006.[4] Valerdi. COSYSMO Working Group. PSM Workshop 2006. [5] Gaffney. COSYSMO-R, 2007.

[6] Valerdi. COSYSMO Working Group. PSM Workshop, 2007.[7] Valerdi, Wang, Roedler, Rieff, Fortune. COSYSMO Reuse Extension. COCOMO Forum, 2007.[8] Wang. COSYSMO Reuse. COCOMO Forum, 2007.[9] Wang, Valerdi, Fortune. COSYSMO Reuse Extension. IEEE, 2008.[10] Fortune, Valerdi. Reuse Considerations. AIAA Space, 2008.[11] Fortune, Valerdi, Wang. State of the Practice. COCOMO Forum, 2008

4


COSYSMO 2.0 Development

• Literature review helped formulate survey questions• Survey results guided proposed COSYSMO 2.0

revisions

Reuse Considerations

Literature Review

Reuse Observations

Industry Survey

Revised Drivers

COSYSMO 2.0

5


State of the Practice SurveyHow does industry handle reuse?

COSYSMO 2.0 Reuse Survey

Reuse Survey Responders

BAE Systems

General Dynamics

Lockheed MartinOrbital Sciences

RaytheonReynolds, Smith, and Hills

Eight responses, representing eleven subject matter experts

6


Survey Results (1)

…it varies

Selected responses• “No formal definitions”• “Use of all or part of systems engineering work products”• “Use of design, pattern, template, handbook, or other engineering

effort that shifts the way engineering is done”• “Use of assets developed or acquired in response to requirements

for one application, in whole or in part to satisfy requirements for another application”

How does your organization define reuse?

Use of existing systems engineering products in a new application

7


Survey Results (2)

Frequency of Reuse of Systems Engineering Artifacts

Requirements

Other

Architecture/Design Models

Test Data/Procedures

COTS Products

Documentation/Templates

Never Occasionally Always

General

Specific

What are the systems engineering artifacts your organization reuses and how frequently are they reused?

8


Survey Results (3)

Artifacts Cited as Most Effective at Providing a Benefit When Reused

Requirements45%

Test Data11%

Documentation22%

Other22%

Which of the artifacts listed above is the most effective at providing a net benefit when reused?

Requirements are the home run of reuse

9


Survey Results (4)

Extent of Reuse of Systems Engineering Artifacts

Transition to Operations

Test and Evaluation

Develop

Conceptualize

Unaware Ad Hoc Planned

To what extent does the reuse of systems engineering artifacts occur?

Similar to distribution of systems engineering effort

10


Survey Results (5)

What are the reasons for reuse successes?– Artifact reused with minimum or no change– Product lines with significant similarities– Requirements management Utilization of personnel with experience on the project that

developed artifact

What are the reasons for reuse failures?– Underestimated modification required for reuse– New requirements placed on a modified product– Customer didn’t modify expectation of risk Lack of knowledge/understanding

11


Survey Results (6)

Promoted Benefits for Systems Engineering Reuse

Schedule21%

Cost29%

Risk19%

Quality15%

Performance16%

What are the most frequently promoted benefits as justification for systems engineering reuse?

Cost benefits implied in others?

12


Survey Results (7)

Frequency of Systems Engineering Reuse Mentioned in an RFP

Seldom50%

Occasionally25%

Always13%

Never13%

How frequently is systems engineering reuse mentioned in an RFP for a new system?

Few instances of systems developed without reuse

13


Survey Results (8)

Scaling of Expected Reuse Savings

Linear25%

Other13%

Non-linear62%

How do the expected savings from reusing systems engineering artifacts scale?

Other is a combination of linear and non-linear

Consensus was non-linearly decreasing as the number of interfaces grows

14


Survey Results (9)

Expected Effort from Reusing a Systems Engineering Artifact

Deleted vs. Managed

Adopted vs. Managed

Adopted vs. Deleted

Modified vs. Managed

Modified vs. Deleted

Modified vs. Adopted

New vs. Managed

New vs. Deleted

New vs. Adopted

New vs. Modified

Less Than About the Same More Than

Evaluate the expected effort from utilizing a systems engineering artifact classified in the first category, compared to the second.

15


Implications to COSYSMO 2.0

• Effects of systems engineering reuse are more than what is captured in the size drivers– Survey results identify personnel, processes, and platform

factors

• Reuse needs to be accounted for in both the size and cost drivers– Size drivers: previously proposed reuse extensions (Valerdi,

Gaffney, Wang)– Cost drivers: newly proposed additional cost drivers (Fortune)

16


Proposed Size Driver Extensions

Modified

Adopted

New 1.0

0

DeletedDeleted

Managed

Reu

se w

eig

ht

Reuse category

New: Artifacts that are completely new

Modified: Artifacts that are inherited, but are tailored

Adopted: Artifacts that are incorporated unmodified, also known as “black box” reuse

Deleted: Artifacts that are removed from a system

Managed: Artifacts that are incorporated unmodified and untested

Based on survey results

17


Proposed Additional Cost Drivers

OrganizationProcesses to capture or implement the reuse of artifacts; repeatable

Domain Applicability

Overlap between the original domain of the artifact and the domain the artifact is being reused within

Technology Comprehension

Availability of documentation or other non-personnel related knowledge assets that provide for or improve the understanding of the technology being addressed in the reused artifact

Reuse Understanding

Completely familiar

Systems engineer directly assisted in the development of the artifact for the original system; continual experience with the artifact; first-hand knowledge of the heritage system is available

Mostly familiarSystems engineer participated in the development of the artifact for the original system; infrequent experience with the artifact

Somewhat familiar

Systems engineer has some familiarity with the artifact and the original system which it was derived from; no first-hand knowledge of the heritage system

Mostly unfamiliar

Systems engineer has experience with similar artifacts but not the current one being reused; limited knowledge of the heritage system

Completely unfamiliar

Systems engineer has no previous experience with the artifact or the system which the artifact was derived from; completely unknown

Artifact Unfamiliarity

Reuse Understanding Artifact Unfamiliarity

18


Conclusion

• Thanks to all who participated in the reuse survey

• Detailed discussion on the COSYSMO 2.0 model to follow in the “Reuse Framework” presentation

1 results of reuse survey jared fortune, usc ricardo valerdi, mit gan wang, bae cosysmo workshop @...

Documents