© 2002 dave lewis & hugh pickerin 1 november 2002 eace cost engineering 2001 maturity survey

© 2002 Dave Lewis & Hugh Pickerin 1November 2002

EACE Cost Engineering 2001 Maturity Survey


Survey Background

• Survey based on CECIM model PAs and BPs• Questionnaire designed & survey carried out by

Anglian Enterprises LTD.• Launched at EACE Frascati May 2001 session• Requested return date 31st July 2001• MS Access database built• SQL used for analysis tasks• Correlation's, means, sensitivities etc analysed• Results charted in MS Excel


Survey Process• Participation free-of-charge to respondents

– respondents encouraged to agree to their identificationin the published results, subject to non-disclosure of specific data

• Identified respondents received:– summary data and full analysis report

(shows organisational ranking versus other respondents)

• Anonymous respondents obtained:– summary report

(shows summary median, mean, mode and max/min values only)

Hence, all respondents obtained a useful analysis tool– aid to internal process-improvement activity

– basis for presentations to senior management


Participants

• 18 organisations participated in survey, including a wide range of European space and aerospace organisations:

BAe Systems

MoD DPA/PFGAstrium - SI

ESA/ESTECAlenia Difesa - Off. Galileo

Datamat SpAAlenia-Marconi Systems


Survey Structure # 1• Survey designed in 2 sections:

– Section 1 captured details of respondents’ organisations:

Respondents Function

Cost Engineering Organisation Details

Organisation Size

Principal Activities

Industry Sector

Constitution

Type of Organisation


Survey Structure # 2• Section 2 invited respondents to rate their

organisational capability, by Base Practice, for each Process Area, according to the CECIM Model:

Continuously Improving5

Quantitatively Controlled4

Process Well-Defined3

Process Planned & Tracked2

Performed Informally1

Not Performed0

[null]N/A

Capability Ratings


Analysis Methodology– Base Practices

• Maximum, Minimum and Modal values recorded for each Base Practice of all 19 Process Areas– total 101 Base Practices analysed

• Corresponding arithmetic mean and median values calculated

• “N/A” responses treated as nulls and discounted for analysis purposes


Benchmark Results – PAs 01 - 10

0

1

2

3

4

5

PA01 PA02 PA03 PA04 PA05 PA06 PA07 PA08 PA09 PA10

MaxMedianModeMeanMin

PROCESS AREAS:

PA 01: Cost Estimating

PA 02: Cost Modelling

PA 03: Cost Control & Analysis

PA 04: VA / VE & Cost Reduction

PA 05: Planning

PA 06: Risk Management

PA 07: Competences Management

PA 08: Define the Process

PA 09: Improve the Process

PA 10: Integrate Disciplines


Benchmark Results – PAs 11 - 19

0

1

2

3

4

5

PA11 PA12 PA13 PA14 PA15 PA16 PA17 PA18 PA19

MaxMedianModeMeanMin

PROCESS AREAS:

PA 11: Ensure Quality

PA 12: Design to Cost & CAIV

PA 13: Supply Chain Management

PA 14: Knowledge Management

PA 15: Capital Asset & Resource Management

PA 16: Business Analysis

PA 17: Business Case Development

PA 18: Audit

PA 19: Cost Allocation


Results Summary #1(survey 1.03) Organisation Structure:

– multi-site, multi-unit organisations reported better capability than monolithic enterprises

(survey 1.04) Organisation Entity Type:

– Private and Public-Quoted companies reported better capability than Multi-Nationals and Non-Profits

(survey 1.05) Industrial Sector:

– overall, no significant differences were observed

(survey 1.06) Principal Activities:

– Manufacturers/Developers reported best capability

(survey 1.07) Number of Persons Employed:

– generally, larger organisations reported better capability


Results Summary #2(survey 1.08) Cost Engineering Headcount:– smallest and largest organisations reported better capability

(survey 1.09a) Dedicated CE Organisation:– dedicated set-ups reported a more consistent and predominantly

better capability than their counterparts(survey 1.09b) Cost Engineering Responsibility:– in organisations without a dedicated CE set-up,

Project/Programme Office predominated(survey 1.10) Respondents’ Role:– Practitioners generally reported the lowest capability; CE

Managers had a more favourable opinion;Other respondents reported the highest capability


Conclusionsare the results significant?

– quantitively, probably not, qualitatively, yes

was the survey worthwhile?Yes!

– deployed a framework enabling participants & others to embark on process improvement activity

– survey respondents and other users can see process improvement opportunities

– provides an additional dimension to CECIM– defined a European Cost Engineering benchmark– illustrates potential of CECIM as the basis for future

benchmarking activity


Recommendationswhere’s the “low-hanging fruit”?• the following Process Areas are recommended targets for

implementation of low-cost improvements:

Capital Asset & Resource ManagementPA 15

Knowledge ManagementPA 14

Ensure CE Process QualityPA 11

Improve the CE ProcessPA 09

action in these areas should yield greatest improvement in overall European CE capability plus greatest leverage of CE resources, thus providing maximum benefit to the implementing organisation


Lessons Learned• inadequate publicity may have restricted participation

lesson: publicise more extensively• the cost - value dichotomy

perception that free participation implies no derived valuelesson: emphasise BENEFITS!

• poor deployment E-deployment was compromised by software version compatibility issues

lesson: ensure electronically-transmitted formats are robust (PDF?)• subjectivity in reporting

lesson: external assessment needed for truly-objective results• survey design ok

general quality of responses good– few incomplete or otherwise spoiled returns

lesson: ensure clear instructions & coherent, user-friendly forms


Copyright Permission:The authors wish to acknowledge the prior work performed under the auspices of the Software Engineering Institute at Carnegie Mellon University, and the grant of permission to use the general CMM architecture, utilise the Capability definitions, reproduce Figures and adapt Process Areas 06, 08, 09, 10 and 11 royalty-free (these portions Copyright © 1995 by Carnegie Mellon University).

CMM and Capability Maturity Model are service marks of Carnegie Mellon University.

Finally, the authors would like to thank the members of the CECIM development team and all survey respondents.


Contacts

• The survey was carried out, on behalf of EACE, by Hugh Pickerin of Anglian Enterprises LTD

• The copyright of CECIM is owned by EACE, you are welcome to use the model in your own organisations but should acknowledge the EACE copyright and all other copyrights.

• All correspondence, particularly comments leading to future improvement of the CECIM are welcome: For advice on implementation of CECIM in your organisation please contact :-

– Cost Engineering Solutions at [email protected]

– or Anglian Enterprises at [email protected]

© 2002 dave lewis & hugh pickerin 1 november 2002 eace cost engineering 2001 maturity survey

Documents