iwsm2014 workshop fpa and quality metrics in contracts (harold van heeringen)

Workshop FPA and quality metrics in contracts

IWSM Mensura 2014

Harold van HeeringenHans Kuijpers

Who is who Drs. Harold van Heeringen

Sogeti Nederland B.V.: Senior Software Cost Engineer; Senior Consultant software metrics.

Nesma: Board member (Marketing & PR); Working groups Benchmarking, COSMIC, FPA in

contract(ing). COSMIC:

IAC member the Netherlands; Chair Benchmarking committee.

FPA and quality metrics in contracts

@haroldveendam

Who is who Ing. Hans Kuijpers

Software Improvement Group Senior Consultant Certified Scope Manager

Nesma: Working groups FPA Quality Metrics in contracts


[email protected]

Background / History Nesma wg FPA in Contract(ing); Start July 2013; 7 people; Assignment:

“Draw up a NESMA document that serves as a guideline (or perhaps even more!) for client organizations and suppliers in order to use function points and quality metrics in RFP’s and in contracts regarding software development and maintenance.”

Guideline for the use of function points in contract(ing).


Workshop Objectives

Present current state of the guideline;

Share content;

Discuss how to get the guideline adopted by

the software industry: both customers and

suppliers;

Formulate ‘to do’ steps and action items.


Why a Guideline

Outsourcing software projects is a challenge;

Organizations struggle to select the best

supplier;

Attempt to use software metrics often fail

Lack of standards / guidelines / best practices;

(Wrong) software metrics are used in a ‘wrong’

way.

Some organizations select a ‘wrong’ supplier,

often resulting in ‘failing’ projects.FPA and quality metrics in contracts

Standish benchmark


Small / Large projects Small projects: in house / easy to estimate

and control; Large projects: often outsourced / hard to

estimate and control.


Low maturity of the industry Customers outsource their IT maintenance

and projects to suppliers; Customers wish to select ‘the right’ partner,

based on objective criteria; Because of a lack of knowledge and low

maturity in software cost engineering practices, often ‘the wrong’ partner is selected;

Even companies that use metrics based on functional size often use these in a wrong way;

Many suppliers don’t measure completed projects and have no idea about their performance.


One example RFP framework question:

What is your price/FP for Java projects of 500 FP?


Scope of the guideline Metrics based on IFPUG/COSMIC/NESMA

function points; Technical quality metrics (ISO 25010); Software development + Software

maintenance; Contracting phase (e.g. RfI / RfP)

Single project / contract Framework: many projects / applications

Contract phase: during development/ maintenance;

All technologies and implementation methods.


Current state Guideline

One guideline resulted in 12 mini guides;1. Mini Guide for Agile development2. Mini Guide for Waterfall development3. Mini Guide for Software Maintenance 4. Mini Guide for Management (ready) 5. Mini Guide for RFP questions (ready) 6. Mini Guide for Functional Quality7. Mini Guide for Pricing Mechanisms8. Mini Guide for Technical Quality (ready) 9. Mini Guide for Assessing Suppliers Performance (ready) 10. Mini Guide for Software Metrics based on FPA (ready) 11. Mini Guide: Requirements for Supplier organizations12. Mini Guide: Requirements for Customer organizations


Do you miss a mini guide we should add ??

Coffee break (15 min) We would like to know your name, company

and role Tick 3 mini guides for more discussion after

the break


☐Demand ☐Supply☐Procurement☐Bid management☐Vendor management☐Supplier management☐Demand management☐Delivery management

☐Account management☐ICT management☐Software Cost

Engineers☐Business

management☐Project management☐Metrics departments

MG for Agile development The silver bullet for killing ‘failing’ projects Manifesto rule 5: Productivity over Velocity Controlling and improving team productivity Why not using story points? How does the customer know all the

requested functionality is ready at the requested end date?


MG for Waterfall development

Metrics in different phases: Requirements/Design/Build/Test/Implementation

Productivity is key in creating successful projects Less effort and cost Better functional quality Better technical quality Adds business value faster

Benchmarking internal and external; The right duration/team size combination is

crucial; What is the effect of changes?


MG for Software Maintenance

80% Of system lifecycle cost is maintenance related

Most Maintenance Contracts are based on the number of defects per application per month Bad code benefits the supplier

This MG focuses on how to demand for maintainable software

*The Influence of Software Maintainability on Issue Handling, MSc thesis, Technical University Delft by Bart Luijten 2010.

MG for Management MG gives answer on:

Why are FPA and quality metrics useful for successful projects? Select and contract a professional partner; Stay in control during the contract phase; Maintenance phase Objective metrics and benchmarking.

The ‘which and how’ questions are answered in the other MGs


MG for RFP questions Customers often ask questions that won’t

enable them to make objective comparison; This MG shows models from literature that are

relevant in composing RFP questions; The effect of duration on an estimate; The effect of selecting an optimistic offer versus

realistic and pessimistic offers; The MG shows possibilities to objectively

compare suppliers’ offers using benchmarking principles and tools;

The MG gives other recommendations as well.


MG for Functional Quality Outsourced projects are often delivered with Can the customer know when the requested

functionality is ready for production? This MG helps in managing expectations

Project will deliver on time with the agreed quality

Useful metrics: Defect potentials Defect Removal Efficiency Trend Bonus/Malus scheme for Early delivery/Low

qualityFPA and quality metrics in contracts

MG for Pricing Mechanisms MG shows the different pricing mechanisms

that are used in the industry, such as Price/FP Fixed price Time and material

All pricing mechanisms have advantages and disadvantages;

MG discusses the way software metrics can be used to overcome the disadvantages;

Example: use productivity targets in TM contracts.


MG for Technical Quality Main contract metrics: time, budget, functional

quality The core of our work is software code The 4th metric for outsourcing is technical

quality Good code quality decreases:

Defect Resolution Time Lifecycle Cost

This MG explains how to demand the 4th metric KPI’s for maintainability, security, reliability

MG for Assessing Suppliers Performance Some organizations outsource all of their

software development and maintenance to 1 supplier; It’s crucial to select the right supplier.

This MG shows a model on how different suppliers can be compared in an objective way.

Basic idea: suppliers send in DCF of x completed projects

The model assesses: Compliancy to requirements of projects; Reality value of data submissions; Productivity/Quality (to each other and to

external Benchmarks)FPA and quality metrics in contracts

MG for Software metrics based on FPA

This MG is a knowledge base for FPA and related metrics

Function Point metrics fit for use in contracts FPA is useful in all lifecycle phases

Project Estimating Supplier selection Project Control: cost, time and quality Benchmarking Maintenance


MG: Requirements for Suppliers

Suppliers need to implement Performance measurement processes;

Using standards is crucial: Project classification; Size measurement (ISO standard); Data collection (standard WBS in effort

registration); Definitions should match benchmarks, for

instance: Hours/FP and Price/FP: which activities included and

which activities excluded per project type? Defects/FP: Which defects included? Delivery Speed: Per manmonth or calendar month?


MG for Requirements for Customers

Business – IT demand – Supplier Customers need to have knowledge in how to

Demand for successful projects Governance the contracts

If not available in your organization invest in it

Using standards is crucial: Estimation based on size measurement (ISO

standard) Technical quality is measurable (ISO standard) Governance by KPI’s Benchmark supplier performance


Brainstorm How can we get the industry to use the MG’s?

Customers? Suppliers?


How to proceed ?!?! Action items? Volunteers to help?


Thanks for participating!

[email protected]

iwsm2014 workshop fpa and quality metrics in contracts (harold van heeringen)

Software

guideline metrics

groups fpa quality metrics

maintenance contracts

waterfall development

technical quality ready9

mini guides1

software maintenance4

development maintenance