Wideband Delphi Method

Introduction

• Predicting the resources required for a software development process

• Software cost and effort estimation will never be an exact science.

• Too many variables - human, technical, environmental, political etc..

• can affect the ultimate cost of software and effort applied to develop it.

• if expectations are not realistic from the beginning of the project, stakeholders will not trust the team or the project manager.

• However, software project estimation can be transformed from a black art to a series of systematic steps that provides estimates for acceptable risks.

Quote

• Watts Humphrey in his book, Managing the Software process, has said, “If you

• don’t know where you are, a map won’t help.” This saying is very relevant while

• dealing with software project estimation. In a software project, unless you are sure that your estimation is accurate, you cannot make much progress.

Fundamental estimation questions

• How much effort is required to complete an activity?

• How much calendar time is needed to complete an activity?

• What is the total cost of an activity?

• Project estimation and scheduling and interleaved management activities

Cost Components

• Hardware and software costs

• Travel and training costs

• Effort costs (the dominant factor in most Project is salaries of engineers involved in the project

• Social and insurance costs

• Effort costs must take overheads into account

• costs of building, heating, lighting

• costs of networking and communications

• costs of shared facilities (e.g. library, staff restaurant, etc.)

Costing and pricing

• Estimates are made to discover the cost, to the developer, of producing a software system

• There is not a simple relationship between the development cost and the price charged to the customer

• Broader organizational, economic, political and

• business considerations influence the price charged

Estimation techniques

• There is no simple way to make an accurate estimate of the effort required to develop a software system

• Initial estimates are based on inadequate information in a user requirements definition

• The software may run on unfamiliar computers or use new technology

• The people in the project may be unknown

• Project cost estimates may be self-fulfilling

• The estimate defines the budget and the product is adjusted to meet the budget

Estimate uncertainty

Estimation – Tools

• Work Breakdown Structure (WBS)

• Dividing into Logical Units/Tasks

• Creating a WBS is a prerequisite for any estimation activity. It enables you to conceptualize an abstract entity, such as a project, into distinct, independent units.

• it gives idea about the size and complexity of the project.

• helps in planning, scheduling, and monitoring a project realistically.

Wideband Delphi Method

• Developed in 40s at RAND Corp as a forecasting tool. Since been

adapted across many industries for estimation.

• A consensus-based estimation technique for estimating effort

• Proven to be a very effective estimation tool, and it lends itself well to

software projects.

• Barry Boehm & John Farquhar originated Wideband variant of Delphi in

70s -Called "wideband" as compared to existing Delphi method, it

involved greater interaction & more communication between

participants.

• Input Work Products:• Vision and scope document, or other documentation that defines the scope of the

work product being estimated• Output Work Products:

• Work breakdown structure (WBS)• List of Assumptions• Effort estimates for each of the tasks in the WBS• Overall, better understanding of the project

• Entry Criteria:• The vision and scope document has been agreed by stakeholders, users, managers,

and engineering team. • The kickoff meeting and estimation session have been scheduled (each at least two

hours).• The project manager and the moderator agree on the goal of the identifying the scope

of the work to be estimated.

Wideband Delphi Process

Wideband Delphi Steps

• Team Selection: Project Manager selects a moderator & an estimation team with 3 to 7 members.

• Kickoff Meeting: The first meeting during which estimation team creates a WBS and discusses assumptions.

• Individual Preparation: After the meeting, each team member creates an effort estimate for each task.

• Estimation Session: The second meeting in which the team revises the estimates as a group and achieves consensus.

• Assemble Tasks: After the estimation session, the project manager summarizes results and reviews them with team

• Review Results: Review the results that have come out from the Estimation Session.

Team Selection• Picking qualified team is important part of generating accurate

estimates.

• TMs must be willing to estimate each task honestly, and should be comfortable working with rest of the team.

• Should be knowledgeable about organization’s needs and past engineering projects to make educated estimates.

• Team should include representatives from each areas of development team: managers, devs, designers, architects, QA, analysts, technical writers, etc

• Moderator should be familiar with the Delphi process, but should not have a stake in the outcome of the session

• PM should avoid Moderator role - should be part of estimation team

• One or more observers - selected stakeholders, users, and managers should be encouraged to attend the meeting

14

Kickoff Meeting

• TMs are given vision, scope & other docs.

• A goal statement for estimation session should be agreed upon by project manager and moderator and distributed to the team before the session.

• Should be no more than a few sentences that describe the scope of the work that is to be estimated

• Ex: Generate estimates for programming and testing the first phase of RedRock Project

• Moderator leads the meeting

15

Kickoff Meeting (Cont’d)• Meeting consists of these activities -

• Moderator explains the Wideband Delphi method to any new estimators.

• If any TM has not yet read vision & scope and supporting docs, the moderator reviews it with the team.

• Moderator reviews goal of estimation session with team, and checks that TMs are knowledgeable to contribute.

• Discusses product being developed & brainstorms assumptions.• Generates a task list consisting of 10 – 20 major tasks. These tasks

represent the top level of work breakdown structure.• Agrees on the units of estimation (days, weeks, pages)

• Disagreement among TMs could result because of• Missing requirements, on which programs or tasks are to be

included• Assumptions

Individual Preparation• After kickoff meeting, moderator writes down

assumptions and tasks generated by team & distributes them.

• TMs independently generates a set of preparation results, that contains • Estimate for each of the tasks

• Any additional tasks that should be included in WBS missed by Team during kickoff meeting.

• Any assumptions that TM made in order to create the estimates

• Any effort related to project overhead (status meetings, reports, vacation, etc) should NOT be taken into account. Should be added to the “Project overhead tasks” section.

• Potential delays, (like certain tasks can’t start until after specific dates) NOT be taken into account. Should be added to the “Calendar waiting time” section.

• Each estimate should be made in terms of effort, not calendar time.

Estimation Form

Estimation Session• Meeting consists of these activities -

• Moderator collects all estimate forms. Estimates are tabulated on whiteboard by plotting the totals on a line

• Estimators read out clarifications & changes to task list written on estimation form. New or changed tasks, discovered assumptions, or questions are raised. Specific estimate times are NOT discussed.

• Team resolves issues or disagreements. Since individual estimate times are not discussed, these disagreements are usually about the tasks themselves, and are often resolved by adding assumptions.

• Estimators revise their individual estimates by filling in “Delta” column on their forms.

19

Assemble Tasks• Project Manager works with Moderator to gather all results

from individual preparation and estimation session. • PM removes redundancies and resolves remaining estimate

differences to generate a final task list, with effort estimates• The assumptions are summarized and added to list. The

Visio doc and other docs are updated with assumptions.• PM should create spreadsheet that lists final estimates that

each person came up with. The spreadsheet should indicate the best-case and worst-case scenarios,

• Any task with an especially wide discrepancy should be marked for further discussion.

• Final task list should be in same format as individual preparation results.

20

Summarized Estimation Results

21

Review Results• Once results are ready PM calls a final meeting to review

the estimation results with the team.

• Goal of meeting is to determine whether the results of the session are sufficient for further planning. The team should determine whether the

• Estimates make sense and if the range is acceptable. They should also examine the final task list to verify that it’s complete. There may be an area that needs to be refined: • For example, a task might need to be broken down into subtasks.

In this case, the team may agree to hold another estimation session to break down those tasks into their own subtasks and estimate each of them.

• This is also a good way to handle any tasks that have a wide discrepancy between the best-case and worst-case scenarios.

22

Pros & ConsPros:

• Very Simple Process

• Consensus-based estimates are often more accurate than individual estimates

• People of would do the work is making estimates

• Among Assumptions are documented, discussed and agreed

Cons:

• Difficult to repeat again and again with different group of experts.

• You can possibly reach consensus on incorrect estimate.

• False sense of confidence is developed at times.

• Experts may be all biased in the same objective direction.

Conclusion

• During last ten years, the Delphi method was used more often especially for national science and technology foresight. Some modifications and methodological improvements have been made, mean while. Nevertheless, one has to be aware of the strengths and weakness of the method so that it cannot be applied in every case.

• Delphi method is better to use as additional method to other research methods.