enterprise risk management and its effective application...

Enterprise Risk Management and its effective application

in developing an organization

Mike Allen Tony Broccolo Jane Petras President Senior Vice President General Manager Michael F. Allen & Associates MJ Electric American Transmission Company www.MFA-A.com [email protected] [email protected]

TABLE OF CONTENTS

1. Executive Summary and session description 2. What is ERM and why should we care?

a. Management Responsibility - Risk exists in every decision we make b. The COSO Cube model for ERM - Portfolio approach to risk management c. The Risk Matrix - Probabilistic risk assessment and valuation d. The Battle Plan - Identification of risk response and control activities e. Management Controls - Communication, monitoring, and reporting

3. How was ERM used to develop the objectives of the Alliance contract?

a. Strategic Objective of the Utility’s Board of Director and Executive leadership of both companies b. Operations Objectives of the utility portfolio management and the contractor’s leadership team c. Reporting Objectives d. Compliance Objectives

4. How did we use ERM to build an organization?

a. Event Identification b. Risk Assessment c. Risk Response d. Control Activities e. Information & Communication f. Monitoring

5. How did we use ERM to build work processes? a. Event Identification b. Risk Assessment c. Risk Response d. Control Activities e. Information & Communication f. Monitoring

http://www.mfa-a.com/

mailto:[email protected]

mailto:[email protected]

6. What specific control tools were used to manage key risks? a. Position Descriptions with detailed roles and responsibilities b. Graded-Approach Methodology c. Probabilistic Risk Matrix d. Detailed Resource-loaded CPM Schedules at both the Project and Program level e. Weekly project team meetings with monthly customer meetings f. Weekly Progress & Production Curves and Job Cost Reports g. Monthly reports with status updates on all known risk drivers and their impacts to project

completion of scope, schedule, and cost h. Internal Project Control Reports i. Job Task Safety Analysis and frequent on-site safety audits

7. What feedback mechanisms were used to evaluate Alliance success?

a. Customer satisfaction measurements, scorecards, and value-added metrics b. Contingency forecasting and month-to-month comparisons of risk matrix c. Forecasting updates and month-to-month comparisons of accuracy and variances

8. What was the customer’s reaction to the ERM application? a. Improved perception as a technical expert b. Improved value as an Alliance partner c. Improved overall satisfaction with program results

9. What were the lessons learned?

a. Some practices that were considered solid indicators of cost were found to be inaccurate b. The ERM approach is valuable planning tool to achieve objectives on multiple levels of an

organization simultaneously/in parallel c. A small group of well-trained practitioners of ERM can positively impact an organization

10. What was the value gained by using ERM?

a. A long-term customer relationship was developed based on a solid foundation of improved communications bond/trust

b. A small group of individuals were able to positively impact a much larger organization c. The customer’s cost was reduced while profitability was increased at the same time d. Much more accurate and dependable data was generated and communicated through the chain

of command in a timely enough manner to allow that data to be used to impact operational decisions that mitigated risks and increased profitability

11. How can ERM help your organization?

a. Every company in every industry deals with risk at multiple levels b. The better risk is managed, the more profitable the bottom line c. ERM can be used to increase customer satisfaction and drive the organization to become more

profitable Attachments:

1. Attachment “A” - COSO “Cube” model for Enterprise Risk Management 2. Attachment “B” – Probabilistic Risk and Opportunity Matrix 3. Attachment “C” – Production Curves 4. Attachment “D” – Alliance Scorecard

CHAPTER 1: Executive Summary and session description This case study presents the successful application of enterprise risk management techniques to develop an organization and work processes between an electric utility and an electrical construction contractor under an alliance contract. Specifically, the Alliance contract was established to execute approximately $250 million of substation and transmission line projects (of which $100 million was construction services). The work was to be performed over a three (3) year period (2007 thru 2009) to upgrade and stabilize the utility’s electrical transmission grid in Northern Wisconsin. Leadership and staff personnel from the utility and the contractor with support from a consultant, formed a team to apply the enterprise risk management (ERM) process to the portfolio of projects to identify, assess, and determine external as well as internal risks related to design, procurement and construction activities. The team set up monitoring, response, and control activities to manage the projects at an individual and portfolio level. This application of the ERM process resulted in a substantial reduction of the total risk expenditures (e.g. projects were managed better and completed on-time and under-budget). Use of the ERM process to build the alliance relationship maximized the information flow between all Alliance Team members, which ultimately resulted in a very satisfied customer and a more mature risk management organization.

CHAPTER 2: ERM – What is it and why should we care? ERM – Enterprise Risk Management starts with the acknowledgement that risk exists in every management decision and activity that is undertaken. How these risks are managed determines the profitability and performance success of the venture. Enterprise Risk Management is a framework for effectively identifying, assessing, and managing risk. This systematic approach to organizing risk as a portfolio creates a “plan” that enables management to effectively deal with uncertainty and the associated risk and opportunity – which ultimately places management in a position to increase profitability and build value. This portfolio approach to risk management is best represented by the “Cube” model created by the Committee of Sponsoring Organizations (COSO) (See Attachment “A”). The ERM process starts at the top with the executive leadership team and flows through the organization. Senior management must determine the company’s appetite for risk, quantify acceptable levels of risk, define the risk management philosophy, and generate the internal environment that supports the strategy to achieve the objectives. Within this internal environment, the objectives must be sorted, categorized, and prioritized. The COSO Cube sorts objectives into four (4) categories: strategic, operations, reporting, and compliance. Strategic objectives are high-level goals set by the President/CEO and the executive management team. Operational objectives center on effective and efficient use of resources. Reporting objectives focus on reliability, accuracy, and timeliness of communications and data necessary for operations to make their decisions. Compliance objectives deal with the minimum reporting requirements to comply with all applicable laws and regulations (e.g. Sarbanes – Oxley). After the objectives have been set and prioritized, then events that can impact the ability to successfully achieve them must be identified. These events include both internal as well as external influences and can have either a positive (opportunity) or negative (risk) impact. All too often the focus is solely on the negative or risk factors that can dilute the bottom-line and the positive impacts or opportunities to increase the profitability are overlooked. The next key step is to assess the risks and opportunities in an organized and systematic manner. The risk assessment should apply a probabilistic approach to two categories: 1) the likelihood of each event, and 2) the potential impact of each event. Identification and assessment of risk leads naturally into the selection of the proper risk response. Responses vary depending on the strategic risk objectives established by the executive management team including: avoiding the risk, accepting the risk, reducing the risk, and/or sharing the risk with other parties. This process of identification, assessment, and selection of the proper response steps generates the necessary “plan” that is so critical to management to allow them to effectively manage the organization and build value.

Control activities must be established to ensure that the risk responses are effectively and consistently carried out. Controls include policies, procedures, processes, and tools used by personnel at all levels of the organization to achieve the consistency, predictability, repeatability, and accuracy necessary for success. Effective communication is essential in every organization. Within the ERM framework project controls act as the conduit to allow the critical and relevant information to be identified, captured, and relayed to the appropriate responsible parties. It also streamlines the timeframe to allow them to use the data to make better business decisions that increase the profitability of the project/program/business unit/division/operating company. The last component of ERM is monitoring. Monitoring can take several forms, but it is essentially accomplished through ongoing management activities such as periodic standardized reports, internal audit evaluations, and third-party assessments. The operational control generated by effective communication, monitoring, and reporting process is what truly impacts an organization’s ability to react to changing conditions. The quicker an organization can react, the higher the probability that the potentially negative risks will be turned into positive opportunities and ultimately improve the bottom-line. ERM is not a linear process, but rather an iterative and interactive process that is multi-directional and flows throughout the organization. The more interaction that occurs within the organization, the higher the probability that efficiency, and thus profitability, will increase as a result of more timely and better-informed business decisions being made at the upper levels of the organization.

CHAPTER 3: How was ERM used to develop the Objectives of the Alliance?

The utility was in the midst of an aggressive capital improvement program which in 2006 had an estimated investment in construction and maintenance projects of approximately $3 billion over a ten year span. The utility’s strategy was to split the capital program into smaller, more manageable, regionally based portfolios. Furthermore, the utility decided to enter into alliance contracts with one construction contractor assigned per portfolio to provide construction services support. This was in contrast to the previous strategy to competitively bid for construction services on each unique project across all regions. The decision to go with an alliance arrangement was based on the following beliefs that an alliance contract would accomplish:

• Share the risks • Share dedication to the customer

• Contribute to mutual growth and profit • Communicate with clarity and purpose

• Promote continuous improvement

For the utility, the management of the risks associated with the organization and work practices of the alliance would be critical to the success of the endeavor. The utility would remain intimately involved in the management of projects but wanted to better integrate the construction contractor into program and project management. To utilize a structured approach such as the ERM framework to implement the alliance relationship made good business sense. In the context of an ERM framework, objectives either existed (strategic) or were created as part of the kickoff of the alliance contract (operations, reporting and compliance). Kickoff meetings were held shortly after contract award and involved representation from both the utility portfolio leadership team and the contractor leadership team. This combined group of leadership personnel became the Alliance Core Team. The result of the kickoff and a few subsequent discussions by the Core Team resulted in the following objectives:

Strategic: developed by utility’s Board of Directors and Executive leadership

• Complete the portfolio of projects safely (as measured by OSHA Incident Frequency rates less than industry averages)

• Complete projects on time (i.e. meet in-service dates) • Complete projects within the budgeted cost (within 10%) • Predict cash flows accurately (within 10% on a three month rolling average)

Operations: developed by the utility portfolio management & contractor leadership

• Work as an integrated project team with utility personnel in the lead role and the contractor personnel as team members

• Complete the specific list of projects assigned to the portfolio and meet the internal and external customer-driven in-service dates

• Do not exceed budgeted or approved dollar amounts • Levelize field resources to make effective and efficient use of the same personnel over a long period of

time • Complete designs well in advance of construction • Balance seasonal and electric system outage considerations

Reporting:

• Generate accurate project costs estimates as early as possible in a project’s life cycle • Report and forecast timely and accurately all work completed, dollars spent, estimates at completion,

progress status, and elimination of risk status at both a portfolio and project level. Compliance:

• Comply with utility’s internal financial and accounting practices • Comply with electrical contractor’s financial and accounting practices.

These objectives were developed to reflect the utility’s appetite for risk and define the utility’s acceptable levels of risk, along with assuring the contractor bought into the objectives. Utilities typically have a low tolerance for risk as is the case in this situation. The objectives were also developed to generate the internal environment needed for the contractor and utility to jointly support the team-approach strategy. Having clarity and understanding of these objectives early in the development of the alliance set the stage for effectively using the ERM process.

CHAPTER 4: How did we use ERM to build an organization?

The organizational development for the Alliance began with weekly meetings of the core team. The purpose of early meetings was for the contractor to understand the utility’s organizational structure and how to build their own staff to integrate seamlessly with them. From the utility’s perspective the meetings served to understand the capabilities of contractor and how they could supplement their own staff. These early meetings resulted in the identification and assessment of the following events (risk and opportunities) which could impact the successful creation of the alliance organization and prevent the alliance from meeting the previously defined objectives: Event Identification and Risk & Opportunity Assessment:

• Alliance contracting was new to both utility and contractor personnel. Personnel on both sides were familiar with a competitive approach to providing services. This is where the owner specifies the work to be done on a project through the preparation of specifications and bid documents, and then multiple contractors bid on the work at a lump sum price. A lump sum bid is traditionally prepared by the contractor with the intent to incorporate all the potential risks into the price while remaining competitive. In contrast, an alliance contract intends that the work is done by the alliance contractor on a time, material and equipment basis. An alliance contract requires trust by the utility that the contractor is providing a fair price to do the work. The contractor prepares an estimate for the alliance work differently than preparing a lump sum bid. The alliance contract has rates with a pre-determined profit amount, and so the contractor must estimate with the intent that risks are transparent and shared with the utility.

• Both the utility and contractor must embrace the team approach. Again an alliance contract is different

than competitively bidding work. When an alliance exists, the contractor is part of the team at the start of the project not just when it comes time to construct. When a project is kicked off, the scope is developed

and the design is performed, the contractor can bring a unique perspective to the project. The opportunity to get the contractor involved and engaged early in project planning is a key to reducing the overall project risk.

• Getting utility and contractor personnel to understand that the objectives were beneficial to the success of

the alliance. The objectives were developed jointly by the core team building upon what the utility’s Executive leadership expected. If the owner wins, then the contractor wins.

In response to the identification and assessment of these events with respect to developing an organization, the core team developed a risk-response strategy. The strategy would be joint efforts by the core team to very clearly define, communicate and train all alliance personnel on roles and responsibilities. The importance of responding to these events cannot be overemphasized. Without first having a strong alliance organization, meeting objectives and developing work practices would be at risk. To address the risk of not developing a strong organization and to ensure that opportunities would be taken, the following initiatives were undertaken: Risk Response Initiatives:

• Creation of clearly defined roles and responsibilities. Position descriptions were developed for each position within the alliance detailing their responsibilities including the communications responsibilities. These positions, while similar to existing roles within the organizations, built on those roles with new specific itemized responsibilities that laid the foundation for change. As the program developed and the alliance needs became more defined, those new responsibilities were overlaid onto the newly defined position descriptions.

• Regular joint training sessions between utility and contractor personnel. Within 6 weeks of the contract

signing, a facilitated, two-day meeting was held between the utility and contractor staffs. The first day concentrated on the benefits and objectives of an alliance. The second day was really the key day in that the personnel broke into mini-teams to start working on developing new work practices to complete projects according to the alliance objectives. Follow-up meetings were scheduled every few months with the same group to reinforce the objectives and share lessons learned.

• Move decision making down to the lowest level possible. This empowers personnel to be accountable for

their actions.

• Incentivize personnel by including the achievement of alliance team objectives in their individual performance goals. Strategic, operational and reporting goals can all be broken down to individual project goals that can be measured on a team and individual basis. When individuals know how their performance will be measured and what they will be “graded” on, they tend to work towards those goals.

With respect to controlling the effectiveness and consistency of the risk responses, the following applied: Control Activities:

• The written roles and responsibilities were only changed with concurrence from the core team.

• The core team with input from the staff, developed agendas, led joint training sessions and facilitated lessons-learned discussions.

• The core team held staff personnel to high expectations related to decision making and accountability.

Decisions were not made for others. Bad or ill-informed decisions were seen as opportunities for learning and improving, not mistakes.

With respect to communication and monitoring, regular informal feedback discussions at a core team level were used as the conduit to assess the effectiveness of the organization as it developed. Discussion questions included: were responsibilities assigned to the right role, would a different role be more effective in carrying out a responsibility, were responsibilities being duplicated especially between a utility individual and a contractor

individual, and how were personnel adapting to their new roles? These discussions resulted in the ongoing enhancements and refinements to the organization. In conclusion the use of ERM techniques resulted in an expedited development of the alliance organization within the first three months of signing the contract. The organization was then the foundation on which work practices could be built. By knowing the objectives, identifying the risks and then acting upon those risks, the exposure of trying something new (i.e. an alliance contract) was substantially eliminated.

CHAPTER 5: How did we use ERM to build work processes? With respect to building work processes at the operations level, the same ERM techniques for developing the organization were used. The identification of events, risk and opportunity assessment, risk response, control activities, communication and monitoring were used by the core team and the organization. This began at the same two-day facilitated meeting to kickoff the alliance at the staff level. Mini-teams were created to review and target existing utility and contractor processes for improvement or modification to an alliance contract. Specific processes targeted were those that would have the biggest impact on achieving the objectives of the alliance. At a high level, the following are the identified events and the risk/opportunity assessment that apply to executing substation and electric transmission projects: Event Identification and Risk & Opportunity Assessment:

• At the beginning of a project the definition of scope plays a major part in determining the overall success. An undefined scope leads to inaccurate estimates and assumptions in how the project will be designed and constructed. Conversely, a well-defined scope that does not significantly change through the life of a project leads to accurate estimated costs from the start and the ability to design and construct to a consistent set of specifications. So the extent of the scope definition can be either a risk or an opportunity.

• Planning a project well includes a schedule for all project activities and mitigation plans for potential risks.

A solid plan leads to better executed projects. With a good plan at the start, the project team will have less rework and less decision-making later in the life of the project. Most of the key decisions are made in the planning phase, and only decisions to address unexpected or unplanned events should need to be made as the project progresses. Committing to spending the time necessary to plan and schedule a project well before starting major activities is a large opportunity.

• Three of the major risks during the construction of electric substation and transmission projects are:

weather conditions, unknown site conditions and transmission system outage constraints. Inclement weather can shut a job site down, decrease work productivity or cause alternate work methods to be used. Unknown site conditions can include worse-than-expected soil conditions or unexpected conditions of existing facilities. Transmission system outage constraints could be cancellation of a planned outage needed to perform construction safely, generated as a result of a failure somewhere else on the system. Should any of these events occur on a project, they can have a significant impact on both cost and schedule. The challenge is on how to accurately estimate the affects and probability of these risks.

• Procurement of materials and equipment is a major activity of transmission and substation projects. This

activity will have schedule and cost impacts on the construction activities of a project if materials and equipment are not available when needed. How to account for the potential risk for late delivery of material and equipment has to be planned.

• Safe field performance can be mitigated by working with a contractor that has a good safety program,

low-level reporting, safety training, error-prevention techniques such as job briefs, field audits, and creating specific safety plans prior to the start of the work. The Alliance established a specific joint safety committee to share lessons learned and develop improved safety initiatives.

• The quality of the design that an engineering firm depicts in drawings, specifications and other documents will greatly impact the safety, cost and schedule of projects. Engineers are typically not constructors so when preparing a design they tend to create what works on paper. This may not necessarily be what works in the field. When constructability problems are encountered in the field (i.e. something doesn’t fit or equipment isn’t available to install the way it was designed), the engineer needs to be consulted to provide a design change. Design changes can result in down time, costly rework or safety incidents.

In response to the identification of these events and an assessment of their likelihood and impact, the following risk responses were developed: Risk Response Initiatives:

• To address the risk of starting a project without clear scope definition, a project challenge process was implemented. This challenge process was a mechanism to review the readiness of a project to move to the design and procurement activities. Personnel not on the project team reviewed the project execution plan and asked questions of the team to determine readiness. If the scope was not adequately defined (i.e. too many assumptions and unknowns), then the project did not proceed.

• An overall schedule, including all project activities, was prepared in detail at the beginning of projects with

involvement from all alliance personnel. Included in the schedule were risk events and associated time to mitigate those events should they actually occur.

• In response to possible weather events (snow, severe cold or spring frost break-up) and system outage

constraints, projects were planned around seasons. This included assessing the probability of normal weather conditions in either the winter or summer or in non-peak loading periods.

• In response to unknown site conditions, more data was collected at a location or facility. For example,

more extensive soil borings were collected or facilities were walked down to compare the as-installed conditions to what was shown on a drawing.

• To prevent late delivery of material and equipment, frequent communication with suppliers directly from

the contractor for status updates was done to allow the contractor to plan around delays.

• To aid in the quality of final designs, constructability reviews of preliminary designs were performed by the contractor. The resulting comments from these reviews were incorporated into final design documents. When the contractor was allowed to review design prints in advance of release for construction, they could make final plans and prepare more accurate estimates for construction earlier.

Control activities of the developed work practices were done similar to that of the organization development. Work practices were prescribed in writing and only changed with approval by the core team. Work practices were rolled out in training meetings with all staff to ensure uniform understanding. Feedback was provided to staff when work practices were not being followed. If work practices were found by staff to be insufficient or inadequate, they were subsequently revised. A significant amount of effort was spent in developing work practices associated with individual project controls. A probabilistic risk matrix tool was developed and used to attack unquantifiable risks on projects. Use of this tool greatly increased the project team’s awareness of the internal and external risks and cost drivers on projects. The net result of this increased awareness was improved risk mitigation plans and improved forecasting of project costs and schedule changes. This tool, as well as many others, is described in Chapter 6. With respect to communication and sharing of information, reporting standards for individual projects, as well as all projects in the portfolio were built. Daily, weekly and monthly reports were created for individual projects. These individual project reports were then summarized into monthly and quarterly reports of total portfolio results. This roll-up approach reduced the cost of reporting and eliminated double-handling of data and the potential to introduce reporting errors.

To facilitate and ensure compliance, various weekly project and monthly program status and project controls review meetings were held with alliance personnel. These meetings increased communications, forced issues to be dealt with in a timely manner, and improved data flow from the field personnel to the core team. As this improved data flow began to gain momentum, improvements in all aspects of the business became more noticeable and began to occur more frequently as the organization began to actively seek out more opportunities to improve work practices. This positive-reinforcement environment was a direct result of the active core team and the well-defined objectives of the alliance.

CHAPTER 6: What specific control tools were used to manage key risks?

Many of the work practices developed were related to control activities on specific projects or in managing the portfolio. Control activities and tools are essential to any successful application of ERM. It was recognized that in a change environment (i.e. alliance contract from bid/build contract), clear direction and communication of roles and responsibilities is critical to achieve success. Therefore, the very first control tool implemented was to issue detailed position descriptions. Position descriptions are a valuable tool in organizing roles and responsibilities in any organization. Key personnel must be able to function in sync and complement each other, so the communication channels and reporting responsibilities are clear, visible to the organization, and used as a basis to hold people accountable. A “graded-approach methodology” that is a by-product of ERM was developed and applied to the Alliance. The ERM model takes risks and organizes it using a portfolio approach to break risks down into manageable pieces for the organization. The graded approach methodology assesses each project within a program and breaks it down into levels of risk exposure. Greater project controls measures are applied as the risk exposure increases. This cost-benefit type approach tailors the amount of project management and project controls resources to match the risk in the program. Thus, more resources and controls were implemented on large risk projects and lower risk projects were allowed to flow through with less control. The Graded-Approach Methodology increased the return on investment by applying project controls where they were needed most. Three levels of projects were established as follows:

• Total Project Cost under $1 million • Total Project Cost between $1 million and $3 million • Total Project cost over $3 million

The development and application of the Probabilistic Risk Matrix for each project, also a by-product of ERM, greatly enhanced the communications and trust-level between the utility and contractor. The risk and opportunity matrix that was developed (Attachment “B”), was used by both the utility and contractor on each project in the alliance program. It helped to drive the project planning and monitoring discussions to a beneficial level well beyond the traditional customer-supplier relationship. The concept of the probabilistic matrix tool is relatively simple:

(1.) identify and quantify as many of the perceived risks on a project as possible, (2.) determine the minimum and maximum dollar impacts for each itemized risk event that would result if the event occurred, and (3.) apply a probability of occurrence to the maximum dollar impact.

The net result is a project contingency that is allocated on a line-item basis to all perceived risks to the project. As each identified risk event passes, its contingency dollars can be either re-allocated or the contingency dollars can be released. Either way, this control tool allows for much greater control over the financial reporting on projects by eliminating large “swings” in margin driven by poor risk management. No project can be effectively managed without a properly maintained schedule tool. For a construction program of this magnitude, a detailed resource-loaded CPM (critical path method) schedule was needed at both the project-level as well as the program level to facilitate planning decisions. Manpower needs were evaluated, scheduled, and discussions held regarding different options to levelize the manpower plan and optimize cost

expenditures. Each month a resource-loaded schedule for the entire program was reviewed with the intent to levelize the required resources needed by the alliance. On large-risk projects, a standard set of weekly performance indicators called Progress & Production Curves (Attachment “C”) were developed and used to communicate weekly progress from the field to the office. The weekly curves were a success as they generated a picture of what was actually occurring on the jobsite. This picture of actual production versus the baseline plan allowed for effective communication to the customer as to what impacts were being incurred, as well as their magnitude; as a result of specific risks actually occurring on the job. These curves greatly improved the forecasting results by providing more real-time data in the form of graphical trend lines. The production curves highlighted trouble areas early enough in the cycle to allow for adjustments and implement corrective measures. Effective communication requires face-to-face discussions on a regular and consistent basis. Weekly internal project team meetings were held to discuss project status, risks, and emergent issues. This frequency greatly enhanced the change tolerance of the organization, as no issues were allowed to fester and generate other issues that distracted focus from the project team goals. These weekly meetings were also good preparation for the monthly meetings, where status was reported and the solutions developed in previous meetings were presented. The end result was a slowly changing organization that was consistently well-informed of all issues, and that was prepared to present solutions ready to be executed upon approval. The nested approach to project reporting (i.e. daily, weekly, monthly and quarterly reports building upon each previous report) improved the communications both within the project team and to the core team. It also reduced the time spent generating reports to meet individual requests, which improved the overall accuracy of reporting and reduced the customer cost associated with the reports. The concept was simple – standardize all weekly production reports, roll-up the weekly reports into a monthly report, and roll-up the monthly reports into a quarterly report. This elimination of re-reporting data streamlined the entire process and eliminated the potential to introduce errors. The monthly reports, as stated, were roll-ups of the weekly reports with the inclusion of a status update on all known risk drivers and their impacts to project completion of scope, schedule, and cost. Again, most conversations centered on the risk matrix and the current status/probabilities of remaining risks. The tools that the contractor used to manage the work were given to the utility and joint training was conducted on how to use them to help better manage the projects. This dramatically increased the communications between all members of the project team and bonded the utility and contractor personnel into a single cohesive team responsible for the program’s success. Finally, the most important set of controls centered on the area of personnel safety. Due to the dangerous nature of this type of construction and the commitment to the safety of all personnel safety was the first topic of discussion at all meetings, in all reports, and in all communications. This proactive approach was coupled with daily job-task safety analysis and frequent on-site safety audits by all personnel involved with the project.

CHAPTER 7: What feedback mechanisms were used to evaluate alliance success?

The first measure of success is customer satisfaction. If the customer is not satisfied, then the potential for future opportunities by the contractor is limited, if not non-existent. As such, customer satisfaction measurements, scorecards, and value-added metrics are important feedback mechanisms to evaluate success both for the customer (i.e. the utility) and the contractor. The Alliance developed an official scorecard (Attachment “D”) that reported key metrics for the Alliance and internal utility success. The metrics included safety, forecasting, schedule adherence, value-added benefits, and customer satisfaction. The specific metrics were chosen to reflect the strategic objective of the alliance and to drive the alliance personnel to achieve the results expected at the utility’s corporate level. In addition, by giving the contractor a vested interest in achieving the utility’s goals, the spirit of cooperation and level of problem-solving increased in intensity and generated additional levels of success. Value-added metrics were established to show additional accomplishments. Specifically, value engineering, cost-savings and cost avoidance ideas that were physically implemented were quantified and included in the scorecard.

Risk management, tracking and forecasting were a focal point of communications and a primary driver of process change to better improve the level of predictability and repeatability of cost forecasting. The tool of choice was the probabilistic risk matrix, which was updated on a monthly basis along with the schedule to generate the cost-to-complete forecast. By allocating risk dollars to specific activities and events that were incorporated into the project schedule, a true integrated resource-loaded plan was updated and maintained. Each month, the updates were compared to the forecast from the previous months to look for trends and variances from the baseline plan. This feedback loop provided data to improve future project plans by tightening the amount of dollars allocated to specific recurrent risk events, and freeing up contingency dollars as soon as possible so they could be re-allocated. This type of constant feedback is critical in large capital construction programs, where the total dollar amount of capital reserved for risk must be kept to a minimum. The impact of this capital requirement alone can be large, and if managed properly, can allow several more projects to be constructed than would normally be possible by freeing up capital dollars on a real-time basis. In this application of ERM, the feedback mechanisms were not just used to evaluate success, but rather they were actually used to fuel the change process and drive the entire organization to another level of success.

CHAPTER 8: What was the customer’s reaction to the ERM application?

The use of ERM techniques to develop the alliance organization, work practices and specific control tools resulted in three of the four strategic objectives being met in the alliance’s first year. The only goal not met was related to safety where the contractor (including subcontractor performance) had more reportable injuries than the top quartile goal set at the start of the alliance. For an undertaking of this size and in such a short time frame, meeting the objectives of on-time project completion and predictable financial results, more than satisfied the utility. The tools that have been developed have greatly increased the overall value of the alliance to the utility by consistently meeting and in many cases exceeding the customer’s needs and expectations. One of the primary benefits that the utility believed an alliance would bring it was the sharing of risk. So it was appropriate that the techniques of an enterprise risk management process were used to develop the alliance. One can say that ERM is just common-sense business management. That may be true. However, ERM provides a structured framework that can be applied to many different business situations. In the case of the utility and contractor alliance, a process was needed to change past business practices quickly, effectively and efficiently. Using ERM gave both parties a common language to use as the alliance developed. ERM simplified the task of overall management of multiple risks and goals. The ERM approach to risk management facilitated a consistent and repeatable process to organize risks and break larger risks down into manageable pieces. By starting at the highest applicable level and then systematically breaking issues down and sorting them into the respective lower levels that could be impacted or that could have an impact on the resolution, a greater application of the appropriate resources was achieved and the overall success was increased. This resulted in a more profitable bottom line, more smoothly-run projects, and ultimately greater satisfaction by the utility. Use of the ERM model streamlined and fast-tracked the ability to implement and execute multiple goals. Again, the portfolio approach to risk management cannot be emphasized enough. A consistent and repeatable methodology to organize, track, and respond to risks at all levels of an organization is critical to the successful management of multiple goals. This is driven by the fact that all levels of the entire organization can be both affected as well as can affect the outcome of any given risk. Thus the whole picture needs to be considered to achieve a consistently profitable bottom line. In the case of the alliance, risks were analyzed at the project, program, and organizational levels. By using this portfolio approach, goals at the organizational level were achieved by monitoring program results which in turn were used to drive project-level responses. Using ERM, a small group of well-trained practitioners positively impacted the organization and drove process improvements at a pace well beyond the norm. The alliance had a positive impact on both organizations. ERM reinforced and proved the validity of the process change improvement objectives that were identified at the outset of the alliance. As these process changes were undertaken and results were achieved, the entire speed of change began to increase as the excitement of success began to spread. In any large organization the ability to

motivate is always a concern, and there is no emotional driver better than the sense of accomplishment and personal achievement. When an organization is properly motivated to achieve goals, the results typically far exceed the initial expectations and lead to other unanticipated improvements – all of which add to the ultimate success and overall profitability of the enterprise.

CHAPTER 9: What were the lessons learned?

While the result at the end of one year in creating the alliance organization and work practices has been mostly positive and beneficial, the activities were not completed without some lessons learned. Some practices were not completely rolled-out with enough training so inconsistency in their implementation was seen. Some of practices were revised a few times because they did not adequately support the objectives to be met. Specifically, initial individual risk assessment methods were not consistent. As a result, cost forecasting was not as accurate and consistent as the core team wanted. Risk assessment varied as a function of the individual project or assigned lead. The methodology for assessing project risks and how to quantify those risks into dollars was not standardized, and was a reason for large swings in project cost forecasts when the risks began to materialize. Forecasting was initially more of a reactive accounting process as opposed to a more proactive estimating process. As the understanding of the probabilistic risk matrix began to spread, its value in predicting issues and minimizing their impact potential began to gain momentum. As a result, the projects began to reflect a more steady and consistent forecasting of costs. Initial cost estimating was also not consistent. The communications feedback loop created using the ERM model allowed for lessons-learned data to be fed back to improve initial project estimates at a much greater pace than had been traditionally experienced. The higher visibility generated by ERM was directly responsible for improvement in the precision of project estimates and change order estimates as projects progressed through the field. Change order estimating was not consistent. Again, the communications feedback loop streamlined by ERM along with the higher visibility that was generated, highlighted the change order estimating process. This allowed for more precision to be applied and better data used to improve forecasts. The weekly Job Progress Report used in forecasting costs-to-complete was subject to fluctuations depending upon project manager input and interpretation. This mainstay report was a tremendous cost-tracking tool, which also provided a forecast of cost-to-complete. This forecast, however, only reflected a “burn-rate” trend in report format. The reporting did not account for whether project conditions would improve or get worse as the project moved forward. As a result this burn-rate “forecast” was open to misinterpretation depending on the individual using the report. The ERM model brought the transparency and visibility to adjust this forecast to reflect the actual production trends in the field, thus yielding a more accurate and precise forecast. During the estimating and job set-up phases, contingency was initially placed in multiple areas and was not tracked with precision. Project managers and estimators had their own personal preference for how much and where to place contingency when preparing project estimate and code of accounts. The transparency and visibility that the ERM model brought to this process, through its organization methodology, greatly improved the project execution consistency by identifying project contingency in a consistent and standardized method. This resulted in more timely and accurate discussions of project issues, their potential financial impact, and ultimately solutions.

Chapter 10: What was the value gained by using ERM? The ERM model is an efficient and proven tool to increase project performance by better managing the risks that management is faced with on a constant basis. The better a company manages its risks, the more efficiently it can respond. ERM allows for a more pronounced focus on using existing resources more efficiently to make business decisions in a timelier manner. This increase in the flow of critical data from the field, in an organized

and more efficient manner, allows the management team to accurately deploy its resources to achieve the most benefit. The value gained by using the ERM approach can be measured on many levels, including but not limited to:

• evaluating the strength of the customer-contractor relationship, • measuring the level of profitability, and • evaluating the tangible process improvements accomplished in the organization that resulted

from the use of the ERM framework. The alliance relationship was developed based on a solid foundation of communication and information flow. This improved flow of data between the organizations led to a mutual respect and trust that bonded the two entities together as projects were executed. The timeliness of this data flow was directly attributable to the application of the ERM framework in organizing the portfolio of risks – both real and perceived – on both a project as well as program level. This relationship eventually developed into the respect and view of the contractor by the utility as a contracting technical expert. Economics also plays a key role assessing the value of using ERM. The ERM framework and its structured approach to organizing and managing risk are directly attributable for the overall financial success of the alliance program. The individual project risk contingency dollars were able to be transferred from project to project within the program portfolio on such a timely basis that the overall capital needs were significantly reduced. None of these savings were generated at the expense of either organization so a win-win scenario developed. The tangible results of the alliance brought through the use of the ERM framework include:

1. Improvement in forecast accuracy and consistency a. New processes were developed to link internal forecasting to external customer reporting. b. Metrics were developed to hold people accountable and measure performance. c. Management follow-up and performance feedback looped back to project team. d. High-level measurement system was also developed to double-check accuracy.

2. Streamlined reporting system

a. Developed a nested system of reports to minimize efforts to meet customer needs. b. Aggressively used existing reports where possible. c. Reports were developed with both internal as well as external formats

i. Internal for project execution and managing the work ii. External for reporting overall program status

3. Graded-Approach applied to project controls

a. Large, complex, high-dollar, high-risk projects received highest level of controls. b. Smaller, flow-work type projects received minimal controls to be cost effective. c. Project control measures were determined long before project was taken to the field. d. Process forced the project teams to plan for risks and establish contingencies. e. Risk Matrix Tool used to increase rigor around, and awareness of, the project.

4. Utility now views contractor as more than a just a solid electrical contractor

a. Proactive management strategy targeted customer goals and key deliverables at leadership as well as project levels to increase their visibility.

b. Interaction with both financial as well as operational teams. c. Focus was on providing contractor planning and management expertise versus raw manpower to

execute projects d. Scorecard developed to report alliance progress in terms of the customer’s metrics that they are

used internally by the customer.

5. Staff is trained to look for continuous improvement a. Every month the staff looks for report improvements b. “What is working versus what is not working” review sessions

Chapter 11: How can ERM help your organization? We all operate in a continuously changing and evolving business world that is driven by the bottom-line. The ERM model is a proven tool to increase efficiency and profitability by better managing the risks that we are faced with on a constant basis. Every company in every industry deals with risk on multiple levels every day. The better a company manages its risks, the more efficiently it can respond, and consequently the better results it will achieve with its decisions. As a natural consequence, the more effective an enterprise is at managing risk, the more profitable it will become. While the risks in different industries will vary, the consequence of better management of those risks does not. We are all measured by the level of results and success we achieve, and this is by no means a static measurement. The goal of business is to constantly improve profitability through both volume increase as well as cost-to-revenue margin percentage. ERM improves the likelihood that an organization can achieve both. The ability to improve margin through volume increase is driven by the level of customer satisfaction that is achieved. Repeat-customer business is a critical component for a successful and sustainable business enterprise. The value of “word-of-mouth” referrals and use of your existing customer base for references is a necessary and “priceless” tool for sales and marketing growth. A satisfied customer is a testament to the organization’s ability to deliver what is promised, and that is a critical factor that is used more and more frequently as a benchmark of performance – and more important as an indicator of the level of risk associated with doing business with the organization. The level of risk perceived by others in doing business with an enterprise will affect its ability to attract new customers, retain the current ones, and ultimately grow the business. The other way to improve profitability is to increase internal efficiency. This approach is often referred to as “cost-cutting improvements”, which only scratches the surface of the potential that the ERM model presents. ERM allows for a more pronounced focus on using existing resources more efficiently to make business decisions in a timelier manner. This increase in the flow of critical data from the field into the organization in an organized and more efficient manner, allows the management team to accurately focus and deploy its resources to achieve the most beneficial return on investment. By making business decisions based on better information, i.e. more timely and accurate data, the profitability and margins of the enterprise will increase faster than its costs, thus achieving profit improvement.

Attachments:

1. Attachment “A” - COSO “Cube” model for Enterprise Risk Management 2. Attachment “B” – Probabilistic Risk and Opportunity Matrix

3. Attachment “C” – Production Curves

4. Attachment “D” – Alliance Scorecard

Attachment “A” -- COSO “Cube” model for Enterprise Risk Management

Risk Event Impact Mitigation Mitigation Tracking Low Impact High Impact

Baseline Estimated Probability of High Impact

Baseline Estimated Dollars at Risk

Current Probability of High

Impact Dolla

Transmission Line

Rock Drilling - Direct EmbedConstruction Cost Increase &

possible schedule dela

Current rs at Risk

ysTrack drilling efforts at each

location $0 $93,280 75% $69,960 50%

Cobble Drilling - Direct EmbedConstruction Cost Increase &


$46,640


location $0 $37,280 75% $27,960 75%

Casing - Direct EmbedConstruction Cost Increase &


$27,960

ys Track efforts at each location $0 $89,123 50% $44,562 25%

Rock Drilling - Concrete Fnds.Construction Cost Increase &


$22,281


location $0 $170,400 60% $101,610 0%

Cobble Drilling - Concrete Fnds.Construction Cost Increase &


$0


location $0 $79,380 33% $26,457 0%

Special Foundation #V01Construction Cost Increase &


$0




$5,995




$3,830


MattingConstruction Cost Increase &


$12,283


Foundation RemobilizationConstruction Cost Increase &


$99,899

ysInsure all anchor bolts on site

prior to start of foundation work.

Insure all easements are in place prior to start of construction.

Track efforts at each location $0 $4,200 0% $0 0% $0

Clearing Remobilization

Construction Cost Increase & possible schedule delays Track efforts at each location

$0 $5,775 0% $0 0% $0

Weather ImpactsConstruction Cost Increase &

possible schedule delays Track efforts at each location $0 $122,475 48% $58,323 48% $58,788

Material Delivery Delays

Construction Cost Increase & possible schedule delays

Work with material suppliers to properly sequence delivery dates

and also have MJE add labor crews upon delivery

Track efforts at each location

$0 $110,600 50% $55,300 50% $55,300 SUBTOTAL

Transmission Line $0 $1,200,538 $523,461 $332,975

Substation

Cobble Drilling - Concrete Fnds.Construction Cost Increase &

possible schedule delaysTrack drilling efforts at each

location $0 $51,675 50% $25,838 50% $25,838

Frost/Frozen Ground/WeatherConstruction Cost Increase &

possible schedule delaysMJE to increase labor support and work overtime as required Track efforts at each location $0 $12,000 0% $0 0% $0

Material Delivery DelaysConstruction Cost Increase &

possible schedule delaysMJE to increase labor support and work overtime as required Track efforts at each location $0 $13,385 75% $10,039 75% $10,039

Outage RiskConstruction Cost Increase &

possible schedule delaysMJE to increase labor support and work overtime as required Track efforts at each location $0 $4,871 50% $2,436 50% $2,436

Caisson Foundation Risk

Construction Cost Increase & possible schedule delays

Engineer to modify specs. Where possible to address field

conditionsTrack efforts at each location

$0 $5,583 0% $0 0% $0

Forced Down TimeConstruction Cost Increase &

possible schedule delaysMJE to increase labor support and work overtime as required Track efforts at each location $0 $16,888 0% $0 0% $0

SUBTOTAL Transmission Line $0 $104,402 $38,312 $38,312

Project Total Risk & Opportunity $0 $1,304,940 $561,772 $371,287

Attachment “B” – Probabilistic Risk and Opportunity Matrix

Attachment “C” – Production Curves

Incentive Plan ATC Scorecard

3.604.50> 3.75–Customer Service

+++++

$1,206K$84K> $250KValue Added5/5N/A6/6Projects Schedule 9%N/A+ 10%Projects Forecast

9.2%3.5%+ 10%90-day Forecast 1.900.00< 3.10Safety (Gate)

Actual (YTD)

Actual (month)

GoalDescription

Attachment “D” – Alliance Scorecard

enterprise risk management and its effective application...

Documents