IEEE TRANSACTIONS ON ENGINEERING MANAGEMENT, VOL. EM-30, NO. 3 , AUGUST 1983 123

A Proposed Project Termination Audit Model DANIEL D . R O M A N

Abstract—Project postmortem analysis, if conducted at all, is gen­erally cursory. Often organizational pressures preclude an independ­ent and comprehensive examination of completed projects. A project audit or outcome evaluation can be extremely constructive and a valuable tool for both the technical and managerial organization. Not only can it help focus on objectives and the accomplishment of those object ives , but it can also provide guidance to the conceptual, forma­tive, and operational phases of future projects.

INTRODUCTION

Project Life Cycle/Phases

Π Ρ Η Ε LIFE CYCLE of most projects can be partitioned into four characteristic phases: a conceptual phase, a formative

phase, a operational phase, and a terminat ion phase. Much thought , research, and writing has been directed to the first three phases; relatively little has been done in examining the fourth phase, project termination, including project or "post­m o r t e m " analysis.

Postmortem Analysis

Project postmortem analysis, if conducted at all, is gener­ally cursory. There may be organizational pressure t o wrap the project up and move on to new areas. Such pressure plus a vague notion that a project has been successful where there has been no significant customer/client complaints, no major organizational disruptions, and reasonable profitability mitigate against incentive for constructive outcome evaluation.

An intelligent and comprehensive examination of com­pleted projects can be instrumental in improving the con­ceptual, formative, and operational project phases for existing and future projects. Was the project terminated because its objectives were accomplished or was it terminated for con­venience, οι worse yet , for failure?

Was the project compatible with organizational objectives? For instance, does the organization have an operational strategy? Did the project complement such strategy? Was it consistent with short- intermediate- and long-range goals?

A project audit or outcome evaluation can be an extremely constructive and valuable tool for bo th the technical and managerial organization. It can help focus on objectives, establish commonality between management and technicians, and lead to reconciliation of relevant factors involved with evaluating the project.

Manuscript received August 16, 1982; revised April 18, 1983. This paper was presented at the IEEE National Engineering Management Conference, Washington, DC, June 1982.

The author is with the School of Government and Business Admini­stration, The George Washington University, Washington, DC 2O052.

AREAS FOR EVALUATION

The Five Areas

To effectively conduct an outcome evaluation, it is sug­gested tha t a comprehensive analysis be conducted in five areas: Technical Objectives, Cost and Budget, Human Re­sources, Project Termination, and the Technical and Manage­rial Project Implications. The elements in each of the five evaluation areas are subject to variation and degree of in­tensity of importance depending on the nature of the proj­ect . What is important is to identify the relevant and strategic project components in order t o provide perspective for proj­ect evaluation and future project management. Too often one overwhelming factor in one of the five evaluation areas can lead to a positive or negative perception of project suc­cess or failure. In all probability, most projects of any mag­ni tude have had both good and bad experiences. Evaluation, in proper perspective, can provide guidance to perpetuate the good aspects, and hopefully, avoid or minimize the bad experiences in future projects.

Technical Objectives

Pos tmor tem evaluation of how well the project's tech­nical objectives were accomplished can be divided into three distinct phases: the technical conceptual phase, the technical operat ional experience, and the ultimate technical product achieved.

In the conceptual phase, the project's technical objectives can be over- or underestimated. There can be a failure t o appreciate the magnitude of the technical problems encoun­tered. At times, underestimating the degree of technical difficulty is done purposely in order to solicit project ap­proval. Postmortem evaluation of the project's conceptual phase should look at such factors as: the project requirements including primary and secondary technical objectives; the pro­posed technical approach; and the supporting technical ser­vices required. Additional concerns include: performance specifications, the reasonableness of quality standards, the identification of facilities and equipment needs, and the achievement of schedule commitments.

Once the project is operational, some of the objectives set in the conceptual phase may change. Final project evaluation geared t o the conceptual objectives may be unrealistic from the technical product originally envisioned. Changes from original technological objectives to the final technical prod­uct can occur for several reasons. The state of the art may shift after the project inception. New and better technological approaches may be dictated in order t o avoid obsolescence

0018-9391/83/0800-0123$01.00 © 1983 IEEE

124 IEEE TRANSACTIONS ON ENGINEERING MANAGEMENT, VOL. EM-30, NO. 3 , AUGUST 1983

and/or to turn out a better end product t h a n was originally planned.

Some other factors which could modify the original tech­nical objectives are: the tendency to overengineer the prod-, uct (this is especially applicable to military and other govern­ment projects); the inclination of technical people to seek per­fection (this is related to the tendency to overengineer); the discovery late in the project's life that there were significant flaws in the original technical approach; pressure from manage­ment for quick and cheap results-this can be related to com­petitive, cost, and time factors; and it is possible that in the initial project startup enthusiasm the scope of technical prob­lems were underestimated and, as a result, overoptimism on the solution of the technical phases of the project colored the original commitment .

The third phase involving the ultimate technical product achieved would entail evaluation of the following: meeting contractual technical objectives; extra or unforseen technical accomplishments which were not anticipated in the conceptual phase; technical objectives not achieved; the success or failure in meeting technical milestone schedules with an explanation thereof; and customer relations and satisfaction.

Cost and Budget

Evaluation of cost and budget achievements in relation to project objectives can, to some extent , be analyzed quanti-filatively. However, cost and budget evaluation should not be oversimplified inasmuch as several factors can affect original cost and budget estimates. Some cost and budget elements can be reasonably controlled, some can be anticipated, and some, such as major changes in the thrust of the project once under­way, may be difficult t o predict.

Assuming there has been no drastic change in the technical aspects of the project it would be meaningful to compare actual expenditures of resources against planned expenditures. Resource expenditures reviewed could include comparison of staff requirement projections against actual persons employ­ment , overhead allocations, facilities and equipment use, sup­porting services required, and external resources contracted for.

Provisions can be made in the postmortem evaluation for some types of cost and budget expenditures which can be an­ticipated, if not directly controlled, such as contingencies for inflation, unforeseen technical difficulties, uncontrollable customer dictated program stretchouts, and added costs re­sulting from the unavailability of material, equipment, or facilities when needed.

Some cost and budget factors that could be difficult to forecast are significant changes in the technical scope of the project resulting from internal technical miscalculations, customer requested changes, and/or a shift in the state of the art which would affect the final technical product if not accomodated. Comparative cost and budget evaluation based on original estimates can be inconclusive if the final project product is quite different from the project product originally planned.

Despite the range of difficulties indicated in evaluating cost and budget performance on a completed project, such evalua­

tion is critical. The other four evaluation areas may score positively; but if costs and budgets are exceeded without some strong reconciliation or mitigating factors, the ability of the organization to survive and continue operations could be jeopardized. Further justification for review in this area would be to look for pat terns. Are there some project ex­penditures which are repeatedly miscalculated? Identifying such areas could be very meaningful in providing better proj­ect managerial control . It could help identify project phases or functional activities where there are tendencies t o con­sistently pad, areas of perennial overoptimism, technical imcompetence, or out and out estimating inepititudes.

Another consideration for cost and budget evaluation would be to review how budgets are derived and costs are experienced. Project budget changes, overhead allocation methods , and actual cost expenditures can be instrumental factors in the ultimate profitability or cost-effectiveness of a project. By carefully comparing actual against planned ex­penditures it might be possible to segregate areas where cost reduction would be possible wi thout impairing the project objectives or the morale of the people assigned to the project. This could lead t o added profitability and organizational flexibility.

Human Resources

In the final analysis, it is people who make or break project objectives. In almost all professions, hard and fast quantifiable standards of excellence for evaluative purposes are virtually impossible to synthesize. Despite subjectivity within the professions, peers generally know who is a good doctor , scientist, engineer, lawyer, or professor. Conceeding subjectiv­ity in evaluating human factors in project management does not diminish the need and the importance of such evaluation.

Several avenues pertaining to human factors can be ex­plored. What were the internal project working relationships? Did the project people interface well with other functional support areas? Were there too few or too many people as­signed to the project? What type of people were assigned? What qualitative and quantitative people support was required external to the immediate project? Was there learning and professional growth? Were the project people professionally flexible, transferable, promotable? What was the ult imate disposition of the people assigned to the project? Were there any significant accomplishments as a result of the project? Was there a participative/consultative environment? What about communication, motivation, delegation? Did the people assigned to the project use their t ime effectively? Can some productivity index on the project be established? If so, to what extent were the project people productive?

Project Termination

Projects can be concluded because the project objectives were met ; the project can be terminated for convenience; or, the project may be terminated for default or failure.

In some projects, achieving technical and/or contractual objectives signals the end of the project. Termination may simply involve the submittal of a final report. On other proj­ects, much remains to be done after the technical work has

ROMAN: PROJECT TERMINATION AUDIT MODEL 125

been completed in order to legally conclude the project. Changes to the contract may continue well past t he ostensible completion because attention must be paid to one or more of the following: work in process, the disposition of special project equipment and purchased raw materials, procurement commitment t o long lead time items, miscellaneous reports, outstanding bills and claims, unconcluded contractual com­mitments , returning the project (area) back to its original condit ion, disposal of finished inventory, disposition of scrap, interorganizational transfers, royalty or patent licensing costs, final settlement negotiations, terminal project audit , and storage costs.

In a multiproject organization even a successfully termi­nated project can require considerable t ime and effort and involve many problems. Termination procedures indicated in the preceeding paragraph requires specialized people. Clean­ing up project residuals is often tedious and unglamorous. Project terminat ion frequently is not given sufficient at tent ion in organizations where the paramount interest is in profession­ally directed technical effort. The orderly phasing out and phasing in of resources should not be neglected and termination procedures definitely should be reviewed and evaluated.

There are also many reasons why projects may not come to successful fruition. It is possible that a project is aborted for convenience before it has achieved its franchised objectives. Termination for convenience can result because organizational resources are constrained and other projects are perceived as having higher priority for available resources. It is also possible that the project may have a high organizational priority but is s topped because of insufficient resources. Other reasons why projects may be terminated for convenience are: new tech­nology may make the present project not feasible; the initial project cost expectations have been exceeded; the potential market has changed and sales prospects in line with costs do not appear encouraging; there are legal implications; com­petitive forces are such as to discourage project cont inuat ion; other investment opportunities promise better re turns for the allocpted resources; the potential contribution t o organiza­tional objectives does not warrant the project investment; there is questionable probability of technical success; and the internal or external political environment may shift with a subsequent diminishing of interest and support for the project.

The worst situation for project termination is where the project activity is stopped because of default. Default most often is related to unsatisfactory technical performance. Termination for default can also result because of: cost over­runs, delivery delinquencies, quality discrepancies, legal viola­t ions, or unsatisfactory material or other substitutions.

In the project termination evaluation phase, termination for convenience may indicate poor planning and termination by default would be presumptive of poor performance. Both situations should, of course, be avoided. In project termina­tion evaluation, whether termination is due t o successful complet ion, convenience, or default, management should determine: What did we learn from the project? What can be salvaged? How can we transfer what we learned on the proj­ect to other projects? Can we improve termination procedures with subsequent reduction in termination time and costs?

Are there any unique termination problems associated wi th this project which might provide guidance for the selection and implementation of future projects?

Managerial Implications

Evaluating a terminated project from a managerial perspec­tive to some extent reviews the effort from the four previous sections as well as providing additional insight. Managerial evaluation while entailing some overlapping can be justified because project performance as perceived by technicians might not be perceived in the same light by management.

A postmortem project evaluation from management 's position would probably focus on such factors as: What was the customer/client reaction? What was the degree of satis­faction? On some type of scale from one t o ten, with one being worst and ten being best, how would the project rate and why? What organizational benefits can be at t r ibuted t o this project? Organizational benefits would be actual or potential follow-on projects, compatibility with organiza­tional goals, planning and strategies, learning or educational knowledge derived by professional and managerial components of the organization, effective use of available resources, de­veloping new or expanded existing physical and human re­sources.

Some additional managerial considerations that might be relevant: t o what extent did this project contribute to the maintenance and development of the professional organiza­tion? Was it completed as planned o r did its project life extend beyond a defensible period? What specific technical innova­tions resulted from the project? Were there any managerial or technical innovations? Were these innovations marketable? What did the project teach us in order to provide guidance for better future performances? Was there any sociological significance attached to the project? Did the project expose potential legal or ethical consequences? What did the project contribute to the organization's present and potential profit­ability?

As indicated earlier in this paper, projects are apt t o vary considerably in technical complexity. All the evaluation factors indicated in this section would not apply to all proj­ects. Some projects might require managerial evaluation in areas not suggested in this section.

THE OUTCOME EVALUATION AUDIT MODEL

Two-Phased Model

In the preceeding five audit areas, several evaluation pos­sibilities were indicated. The Outcome Evaluation Audit Model is a two-phased model. Phase one is procedural or descriptive to provide guidance for identification of relevant evaluation factors. Phase two is prescriptive in that once the pertinent evaluation factors are identified, problem areas can be isolated and corrective action, where applicable, can be initiated.

Also, improvements to planning can be facilitated t o avoid similar problems on future projects. The mechanics described are only suggestive. Each project environment would probably require innovative modifications.

126 IEEE TRANSACTIONS ON ENGINEERING MANAGEMENT, VOL. EM-30, NO. 3 , AUGUST 1983

Phase 1-Prepare an Audit of the Following: 1) Technical Objectives:

a) primary objectives, b ) secondary objectives, c) the proposed/actual technical methodology, d ) history of changes to performance specifications, e ) e tc .

2) Cost and Budget: a) actual versus planned expenditures, b ) adequacy of contingency allowances, c) profitability/cost-effectiveness, d ) e tc .

3) Human Resources: a) internal working relationships, b ) interface with other projects, c) professional development, d ) e tc .

4 ) Project Termination: a) completion, convenience, or default, b) termination procedures and problems, c) legal implications, d ) e tc .

5) Managerial Implications: a) customer/client reaction, b) contribution t o organizational objectives, c) development of corporate resources, d ) e tc .

Phase 2

Procedure to Implement Outcome Evaluation Audit Model

It is impor tant to have a organizational vehicle to imple­men t the ou tcome evaluation audit model. There is precedent in tha t most technical organizations have change control re­view procedures. A change control review board frequently has representatives from engineering, product ion, and quality control . Other functional areas on occasion may also be repre­sented. It is proposed that a representative review board be established for postmortem evaluation.

The proposed review board should normally consist of representatives from the technical division, product ion, quality control , finance, contracts administration, and management. Membership representation can vary to reflect functional participation due to the nature of the project. It is also sug­gested that assignment to the termination review board be rota ted and tha t participants on the board should no t have been directly involved with the project(s) being reviewed. Neu­trality of board members should encourage objectivity in reviewing the project and help avoid a partisan, advocate, or defensive approach. Rotating membership on the board or panel could, in addition, provide added insight and educational benefits to participants which can be carried over t o their own involvement with other projects.

CONCLUSIONS In an organization where there are many and, at times,

diverse projects, the tendency is to complete the project and

Frequency: Check One

Factor No Problem Periodic Problem

Constant Problem

Not Important Important

Very Important Explanation of the

Problem Possible Solution(s)

Technical objective

Methodology X

very important tendency to miscalculate magnitude of technical effort and difficulty

Greater in-depth analysis of problem required. More study and coordination of technical aspects. Inadequate technical preparation can be overcome by more extensive checking on technical approaches.

Cost and budget

Expenditures X

very important cost overrun Tighter estimating. Estimating each project work commitment to an item and phase level to provide better project control and detect problems quickly.

Human resources

Low productivity

X

very important inadequate planning; unavoidable support resources

Frequent failures on meeting schedules can be avoided by improved interface with other project managers to determine common facility resource requirements in proper time frame.

Project termination

X

important failure to expedite contractually required termination procedures

Develop specialized staff familar with legal aspects of termination procedures. To relieve project people who can more quickly be reassigned to ongoing technical projects.

Managerial implications

Contribution to organizational objective

X

very important project not compatible with growth objectives or available physical and human resources

Avoid temptation to take business and commit resources to projects not consistent with organizational interests and resources. Establish a multifunction project review board to evaluate new projects.

ROMAN: PROJECT TERMINATION AUDIT MODEL 127

move into something new and, hopefully, professionally stimulating. By the t ime the project has evolved into its final phases people assigned to the project may be bored and anx­ious t o move to another project since the professionally challenging work has already been accomplished. The pos­sibility of an enlighting and more than casual postmortem evaluation promises little professional stimulation. Worse ye t , a postmortem audit can disintegrate in to a finger point­ing accusing exercise. The purpose of a pos tmor tem review is not accusation but for professional and organizational per­formance improvement.

Project evaluation should be holistic. This paper has stressed five significant audit areas. It has also been mentioned that one overwhelming factor can subvert many other pertinent and important factors. It is urged that any postmortem project evaluation procedure involve representatives from the different organizational elements which have contr ibuted to the project . This is very important in order to minimize distort ion based on narrow functional orientat ion. For ex­ample, in evaluation, technical project people can be pleased

with accomplishments t o the exclusion of other factors which are essential elements to project success. Even though techni­cal objectives have been achieved management can be dis­satisfied for a variety of reasons such as an excessive cost, low or no profitability, little potential for follow-on, questionable contribution t o organizational objectives, and poor utilization of scarce resources. Conversely, management might be satisfied but the project professionals might feel the results are not technically as good or possible. There is always the conflict between the managerial pressure for fast results and minimal expenditures and the professionals aim for technical per­fection.

Lessons learned from a comprehensive evaluation audit can lead to improvement of organizational productivity and this, in turn, can lead t o greater operational flexibility and provide an expanded operational base. With proper com­munications and people education it should become apparent that if the organization prospers and thrives the people within the organization will also benefit professionally and economi­cally.