3rd party independant reviews

The gateway review process

a quite success story of using a quality assurance process to improve the delivery of high risk projects

…….and more generally, some lessons from around the globe

Nick Pelham

But first…..a case study in asset management : project vs operations

The iron triangle of project management

TimeCost

Quality

Sydney Opera House, NSW (1952)

• Time planning began in the 1940s

• The need - “a suitable venue for large theatrical productions”

architectural design competition awarded in 1957 (Jorn Utzon).

construction started in 2 March 1959

Original completion date – 26 Jan 1963

Opened in 20 October 1973• Stage 1 (upper podium –> „59-63)• Stage 2 (roof -> „63-67)

Construction of the Opera HouseV

original estimate – £3.5m (~$7m) (1957)

final cost ~ $98.8 - $102m (1973)

since 1973... the first upgrade - $120m (1988-98) the second upgrade - $ 66.59m (2002)

total construction and refurbishment – (1957-2002)

$374.4m

+ „Venue Improvement Program‟ (2001-09) $69m

You all have your views……

Quality

TimeCost

1400% + 3½yrs ->14+ yrs

• Average cost per year over past 5 years

• $121m pa• Bldg dev and expenses $ 31.5m• Expenses $ 89.5m

• + „Venue Improvement Program‟ (2001-2009) - $69m($7.7m per year)

• More likely average cost per year

$128.7m pa

Running of the Opera HouseV

2011 NSW Audit

SOH needs a $1.1b upgrade to its facilities to remain viable and to address access and safety concerns:

• flawed acoustics• limited accessibility• limited seating and restricted spaces• shortcomings in OHS

But………

Construction

• $102m (1973) $102m

Upgrades (1988-2009)

• Upgrade 1 (1988-1998) $ 120m• Upgrade 2 (2002) $ 66.59m• Venue Improvement Program (2001-2009) $ 69m

$ 255.59m

• Upgrade 3 (2014- 2023) $1,100m

Running the opera house

over next 10 years $1, 326.6m

Summary

So what…….

Rule of thumb

1 - 4 - 20

1. Gate review processes

Origins of the ‘gate’ concept

underlying idea behind Gate Reviews can be traced to research and development in the private sector in the 1980s and is widely attributed to work Dr R. G. Cooper (Canada).

The terms used to describe such reviews include ‘Stage‐Gates’, ‘Gateway Reviews’ etc….

The approach remains strongly identified with product innovation and the introduction of new products to the market by private companies.

In public sector organisations

The Gateway Review Process in Australia began in 2003 and is modelled on the UK’s Office of Government Commerce (OGC) Gateway™ Process that was originally designed to address failures in information technologyprojects

Developed by Sir Peter Gershon in the UK in Feb 2001

Project performance-cost overrun by percentage (based on Flyvbjerg, 2003)

The Gateway Review Process

Gate 1: Strategic Assessment- Confirm business strategy and need

Gate 2: Business Case- Confirm business case is robust

Gate 3: Readiness for Market- Confirm procurement method and sources of supply

Gate 4: Tender Decision- Confirm investment decision

Gate 5: Readiness for Service- Confirm commissioning plans are in line with business strategy and

industry best practice

Gate 6: Benefits Evaluation- Assess project delivery effectiveness

Opportunity to influence project outcomes

Strategic

assessment

Procurement

Strategy

Competitive

Procurement

Award and

implement

contract

Manage Contract/facility

to deliver services

Develop

Business Case

Benefits

realisationConstruction Readiness

for Service

Needs

analysis

Business

justification

Procurement

strategy

Investment

decision

Whole

of Life

Expenditure

0%

100%

Time

What we’ve learnt.........

BENEFIT STRATEGIC

INTERVENTIONSPROBLEM

Improved liveability

around the market

KPI 1: Reduction in CO2 and

noise emissions

KPI 2: Improved health of

residents

Restrict truck

movements to daylight

hours

Remove trucks from

residential streets

Relocate residences

outside market area

Increasing road

freight in residential

streets is causing

declining health in

people who live near

the market

Starting at the start…..defining the Problem

Focus on a Case for the Change (business case)

• No executive summary

• No genuine link to strategic priorities

• Lack of specificity as to the ‘need’ or proposed solution –aka solutions looking for problems

• Lack of clarity of timing, cost and benefits

• Use of diagrams or tables which are unrelated to the text or hard to interpret

• Obvious inconsistency both numerical and with the detail of the business case

Traps for new players or poor past efforts:

The gateway review process ‘adds value’

UK –>

•2008 2-4% cost avoidance *

Australia

• 2009 3-5% cost avoidance

• 2010 22.4% time benefit, 29.3% cost benefit (Uni Melb)

* If all recommendations are implemented as outlined in the gateway review report

Assurance provided from the Review Process

Judging success

• Large projects often best judged with the benefit of hindsight

• Time and cost are useful yardsticks for success- ………. as is function and stakeholder acceptance

• Large projects are inherently delivered in a high risk environment

• Performance of capital projects are largely determined by time and cost

- Remember Sydney Opera House– Project outcomes

– Upgrade costs

Other jurisdictions

Whole of Life analysis – individual projects

Australia

1. Managing complex governance arrangements across agencies and jurisdictions

2. Project inception, approvals and funding

3. Risks associated with implementing projects in tight timeframes

4. Establishing a benefits lead approach

New Zealand

1. Business case

2. Risk and issues management

3. Program/project management

4. Stakeholder management

5. Resourcing

6. Governance

7. Program/Project planning

8. Management of change

9. Sourcing strategies

10. Dependency management

2. Structural deficiencies

‘Reviews’ of the reviews

Audits in the UK and Australia

2010

Audit of the Efficiency and Reform Group’s role in improving public sector value for money

Ombudsman investigation in to ICT enabled projects

Assurance for high risk projects

Assurance for major projects

Gate Reviews for Defence Capital Acquisition Projects

Administration of the Gateway Review Process

Parliamentary enquiries

2011 2012

What was found (+)

+…..a central quality assurance tool which assists in the delivery of high risk and complex projects…..

+…..valued by stakeholders for the benefits that result from preparing for a review as well as the impact of their findings on reducing project risks and avoiding costs.

however……

Improvements can be made (-)• Central assurance arrangements :

• were inadequate

• can be effective in isolation, but are more valuable when designed and operated as an integrated system.

• A good assurance system should highlight any breach of time, cost and quality control limits (as agreed when approving the business case) and trigger appropriate intervention.

•Improved visibility of project performance, tracked at the portfolio level, should lead to improved decisions across projects and departments.

•Assurance should transfer lessons across the portfolio.

effective integrated assurance systems

Four core components :

1. Plan assurance requirements across the portfolio.

2. Perform assurance reviews.

3. Report the findings and lessons.

4. Control and improve performance by using information in the system.

SO what was missing

•Focus on individual project success and not on the portfolio

•Reliance on ‘the project’ to rectify their issues and not that of the portfolio

Responses and Actions going forward

• centralisation and coordinated • “Major Projects Authorities”

• stricter approvals – new projects not considered unless assurance processes have been followed.

• Increased governance and oversight:

– actively reviewing the top x projects.

– databases of the riskiest projects/ programmes (technical risk, lifetime spend and priority); and

– increased powers to stop or re-scope failing projects, or additional help to boost management capability.

Summary (1 more slide to go)

• A number of benefits have emerged from implementing gate reviews

-Increased assurance that the expected service delivery outcomes will result in budget/costs are more likely to be achieved

-Project Sponsor and Senior Responsible Owners are important

-Lessons learnt systematically captured and analysed to track hotspots/ weaknesses

• Integrated, systematic and holistic management approaches are required to deliver high risk projects

-time and cost no longer provide decision makers with adequate decision evidence

-In modern project management, the importance of multi dimensional performance is being realised

Emerging issues

1. Convergence of peer reviews vs performance audit

- points of difference and ‘value’ offered by each

-assurance : documentation and interviews = assurance snapshot

-audit : standards and performance + ‘deep dives’ = governance check

2. Capital + Sustaining Infrastructure

- time and cost yardstick

- functionality and fit-for-purpose

- definitions of success + horizons changing

nlpelham@hotmail.com