Past Performance Information

John SavickyArizona State University

May 2013

2

PIPS Best-Value Process

Identification of PotentialBest-Value

PHASE 1

Pre Planningand

Quality Control

PHASE 2

Award | Measurement &Documentation

PHASE 3

3

PIPS Has Multiple Filters• No single step will complete minimize

all risk. Process has several steps that act as filters.

• Filters do not limit competition, but make it difficult for non-performers to be competitive.

• Looking for dominant information

4

What is the Objective of PPI?• PPI should not be used as the sole factor to ensure that the vendor can

perform on a current project.

• Communicate to the vendors that the owner will purchase based on value (performance and price).

• Identify if a vendor has capability to perform.

• Encourage vendor to use and partner with the best people / best subs

4

5

How Should We Collect PPI?1.Survey all of the vendors past

projects?2.Survey all projects done for just your

organization?3.Select a random number of past

projects (good and bad)?4.Survey the last ten projects?5.Survey all the projects completed in

the last 2 years?6.Survey only projects that are similar

to the type of project being procured?

6

In PIPS, Vendors Are Allowed To Only Select Their Best

Past Projects

7

Is This Any Different Than The Traditional Approach?

8

What Questions Should We Ask?• Ability to manage cost• Ability to manage schedule• Ability to communicate• Ability to manage subcontractors• Ability to follow facility rules• Ability to minimize change orders• Ability to minimize claims• Ability to minimize punch list items• Knowledge of codes• Management abilities• Quality of submittals• Quality of workmanship• Professionalism

8

Response time to emergencies Response time to warranty work Knowledge of regulations Willingness to do value engineering Contractor’s level of honesty Overall construction knowledge Overall construction quality Overall performance of contractor Customer public relations Your overall customer satisfaction Was job completed on time Skill of craftsmen

9

What Questions Should We Ask?

9

10

Statistical Analysis• Objective:

– Determine if survey questionnaire could be minimized

• Methodology: – Analyzed 175 contractor PPI databases (GC, Elec, Fire Alarm)– Analyzed 37 criteria– Perform statistical analysis

• Results:– Results indicated 8 criteria would generate same results as asking 37– Re-tested 8 criteria on 50 randomly selected contractors. – Further tested on 29 randomly selected HVAC, Plumbing, and Painting

vendors.

10

11

Survey Form

11

12

Collecting Surveys

• Option 1 – Client collects surveys on all vendors

• Option 2 – Client hires a third party to collect surveys

• Option 3 – Vendors collect their own surveys

13

PPI Analysis• If Client collects surveys, be prepared for additional work

– Must call to confirm– Must have several machines to accept surveys / paper– Must have people to contact regarding surveys– Must enter data and double check for errors– City of Peoria = over 8,000 surveys– University of Minnesota = over 4,000 surveys

14

In PIPS, Vendors Are Required to Survey, Collect,

and Submit Their Own Surveys

15

DETAILED STEPS

Past Performance Information• PPI will be collected on the critical Entities:

– Firm– Project Manager (Individual)– Key Subs or Suppliers

• Each Entity must prepare and submit a list of references, customer surveys, and Past Performance Information Scores

16

VendorENTITYPrepare and Send Survey Questionnaires to Past Clients

Step 2

Step 3Collect/Receive Completed Surveys

Prepare Reference ListStep 1

Enter data into Reference ListStep 4

Package all material (Reference List and Surveys) and SubmitStep 5

Reference List • Maximum number of references that can be submitted is 10 for each Entity.

• The past projects must be installed and operational.

• The reference list must contain different projects.

• The reference list must contain different clients.

• The client must complete the survey (no consultants or third parties)

17

Survey Questionnaire • Each Entity must prepare, send out, and

collect survey questionnaires.

• All returned survey MUST be evaluated AND signed by the client. If a survey is not signed, it will NOT be counted or considered.

• Returned surveys must be submitted with Proposer's proposal (Proposer should keep a copy of all returned surveys).

18

PPI Score• Proposer shall input the survey scores,

and then average all of the responses to obtain the “Overall Average Score”.

• Proposer shall also count the “Total Number of Returned Surveys”.

• The Client may contact the reference to clarify or to obtain additional information. If the reference cannot be contacted, the survey may be deleted.

19

No Criteria

Surv

ey 1

Surv

ey 2

Surv

ey 3

Surv

ey 4

Surv

ey 5

Average

1Satisfaction with the staff assigned to the project

10 9 10 10 9.8

2Ability to meet your goals and expectations

10 9 10 10 9.8

3Ability to integrate and interface with any existing systems

10 9 9 10 9.5

4Ability to communicate and document system functionality

10 9 9 10 9.5

5 Overall quality of the installed product 10 9 9 10 9.5

6Overall quality of the company’s service

10 9 10 10 9.8

7 Overall customer satisfaction 10 9 10 10 9.8

8Ease of implementing new requirements into the system

10 9 9.5

9.6

4

Overall Average Score:

Total Number of Surveys Returned:

20

Summary• PPI is only one step in the process.

• PPI is not used as the sole factor to determine future performance.

• The PBSRG has collected PPI in many different ways:– Sent surveys to past owners, called owners,

collected data– Allowed vendors to send surveys, but collected the

data– Allowed vendors to send and collect surveys

• The goal is to be efficient. Research has shown that the most optimal way to collect PPI is to place the vendor at risk and allow them to be completely responsible for their PPI.

21

Comments / Questions

W W W . P B S R G . C O M