1

Through the Looking Glass - How to do a Post Project Review

Barbara Purchia

February 20, 2001

Rational Softwarebpurchia@rational.com

2

Overview

• What is a Post Project Review?

• When to do it

• How to do it

• Post Project Review Meeting

• Real life problems

• What have we found so far

• Summary

Agenda

3

A Rose by Any Other Name

• Post Project Review (PPR)

• Project Retrospective

• Project Legacy

• Project Summary

• Post Mortem

• Post Partum

What Are They?

4

What is a Post Project Review?

• A summary of project development, start to finish• A brief description and analysis of

– Project activities– Results– Lessons learned– Recommendations

• An improvement tool– NOT a record of failures

What Are They?

5

Why Should We Do a Post Project Review?

• Learn from project experiences

• Retain knowledge for future projects

• Understand how your project was done

• Identify areas to improve

• Understand successes

• Provide team closure

What Are They?

6

When Do I Do One?

• For completed, terminated, or cancelled projects

– No more than 30 days after project end

– NOT before project release

– When all critical participants can attend

• During the project

– After a major project milestone

– At designated schedule points

– When things are not going well

???

When to Do Them

7

What’s Involved?

• Collect the information

– Technical

– Managerial

– Metrics

– Schedules

• Anything kept during the project

– Manager or project logs

– Emails

– ...

How To Do Them

8

What’s the Deliverable?

• Concise documented information summary

– Development activities

– Project metrics

– Estimates and actuals

– Technical and managerial lessons learned

– Recommendations for future projects

• Focused only on the project and the processes

How To Do Them

9

How Do I Start?

• Hold a meeting including

– ALL participating functions

– Project management team

– Affected functions

AND/OR

• Request input via email

– Remote sites

– Managerial lessons learned

– Filtered to be unattributable

How To Do Them

10

What Is a PPR Meeting?

• A tool for reviewing and analyzing the project

• Promote discussion on improvements

• Involve a critical mass of those directly involved

• Promote discussion of all project areas– Deliverables

– Team dynamics

– Processes

• Provide input for the PPR document

• Provide closure for the project team

How To Do Them

11

What Doesn’t the PPR Meeting Do?

• Assign blame to any individual or group

• Provide an exhaustive study of the project

• Provide solutions to all issues

• Shoot the messenger

How To Do Them

12

How to Hold a PPR Meeting

• Use a facilitator

• Set up a firm time

• Use an agenda

• Ensure Senior Management support/sponsorship

• Keep the focus on the process not the people!!!

How To Do Them

13

How to Plan for the PPR Meeting

• Pick the meeting date

• Identify attendees and ensure availability

• Book the room in advance

• Prepare an agenda

• If using a questionnaire, review the questions and

tailor it for the project

• Send meeting announcement

How To Do Them

14

Roles and Responsibilities

• Senior Management– Ensure that a PPR is done

• PPR Sponsor– Call the PPR meeting

– Ensure that information gathered

– Produce and post the PPR document

– Use the data

• Project team members– Attend the meeting and participate

– Use the data

• Facilitator– Non-project member

– Help plan and organize meeting

– Ensure meeting runs smoothly

• Administrator– Capture the issues and

recommendations

How To Do Them

15

PPR Meeting Attendees

• Managers• Team Members• Project• Program• Product• QA/QE• Development

• Documentation• International• Beta• Service• UI• Manufacturing• ...

All Project Participants

How To Do Them

16

Facilitator Preparation

• For each group, work with the sponsor to

– Understand the environment

– Understand the culture

– Understand the people

– Understand any previous PPR activities

How To Do Them

17

Sponsor Preparation

• Produce a brief project refresher

– Time line of key milestones and/or significant activities

• Review the PPR questionnaire with the Facilitator

• Identify anticipated issue categories with the Facilitator (optional)

• Review any previous PPRs

– Documents

– Improvement activities

How To Do Them

Start

75 MPH Milestone

18

Participant Preparation

• Think about the project

• Review the questionnaire

• Review meeting minutes

• Review emails, documentation, ..

How To Do Them

19

Room Preparation

• Done by the facilitator

• Ensure that meeting aids are in the room

– Flip charts

– Markers

– Post-its

– Tape or tacks

– Pens

• Bring candy, donuts, or have lunch available after the

meeting (optional)

How To Do Them

20

PPR Meeting Logistics

• Split into two parts

– Looking backwards

– Looking ahead

The PRR Meeting

21

Example Two Hour Meeting Agenda

• Introduction and Groundrules (5 minutes)

• Project refresher (10-15 minutes)

• What Went Well and Issues Gathering (40 minutes)

• Prioritization (10 minutes)

• Brainstorm solutions for top 3 issues (40 minutes)

• Meeting Closure (5-10 minutes)

The PRR Meeting

22

Groundrules

• Be polite and gracious

• Focus on problems and process

• No finger pointing

• No assigning blame

• No “You’s”

• Let each person have a turn

• Avoid defensiveness

• Respect everyone’s perspective

Off with her

head!!!

Off with his

head!!!

NOT!

The PRR Meeting

23

Project Refresher Example

Milestone Planned Actual

VX.0 Development Starts 7/98

Requirements complete 9/9/98 9/29/98

Preliminary Project Plan complete 10/4/98 10/27/98

Functional Specs complete 10/23/98 1/19/99

Schema Freeze 10/23/98 1/19/99

Commitment Meeting 11/03/98 11/17/98

UI Freeze 12/31/98 1/20/99

Feature Freeze 1/15/99 2/3/99

Beta Test Start 1/15/99 3/1/99

Code Freeze 3/9/99 5/29/99

Release to Manufacturing 3/23/99 6/15/99

The PRR Meeting

24

Barb’s Lessons Learned

What Went Well and Issues Gathering

• Split into separate activities

• Use Post-its

• What Went Well

– Use flip chart sheet(s)

– Put “What Went Well” Header on flip chart sheet

– Write down positives on Post-its

– Read in round robin fashion until done

– Facilitator puts on flip chart sheet

– Group any duplicates

The PRR Meeting

25

Barb’s Lessons Learned

The PRR Meeting

What Went Well and Issues Gathering (Continued)

• Issues

– Use flip chart sheets

– Write potential issue categories on top of blank sheets

– Have several blank sheets

– Tape to wall

– Write down issues on Post-its

– Read in round robin fashion until done

– Facilitator puts on flip chart sheet

– Group any duplicates

26

Post-It Rules

• One item per Post-It

• Write as many Post-Its as you want

• Write enough so that each idea stands on its own

• Read 1 Post-It per person until we are done

• Identify duplicates and pass them to the facilitator

• One item per Post-It

The PRR Meeting

27

Issue Prioritization

• Everyone checks issues

– Ensure grouped correctly

– Ensure duplicates collected correctly

• Everyone gets 3 votes

– Used however person feels

• Total all issues

• Identify top 3 issues

– Check in with group to see if agree

The PRR Meeting

Last minute changes to requirements (8)

Finish critical components sooner (4)

More Beta Test sites (4)

28

Brainstorm Solutions for Top 3 IssuesTechnique 1

• Use flip chart sheets– Identify the issue and page number on top right

– Write issue on top of flip chart sheet

• Write down all solutions– Get clarification as needed

– Check in to see if captured correctly

• Spend 10 - 15 minutes per issue

• Check in if energy low

The PRR Meeting

29

Brainstorm Solutions for Top 3 IssuesTechnique 2

• Identify volunteer facilitators for the top issues– Include similar items

• Break up into groups

• Volunteer facilitators should use flip chart sheets– Identify the issue and page number on top right

– Write issue on top of flip chart sheet

• Write down all solutions– Get clarification as needed

– Check in to see if captured correctly

The PRR Meeting

30

Meeting Closure

• Thank people for attending

• Answer any questions

• Re-iterate the next steps

• Thank people again

The PRR Meeting

THANKS!!

31

Facilitator Tips

• Protect each speaker

• Pay attention to the speaker

• Keep the meeting positive

• Don’t filter the information

• Let people finish their thoughts

• Help people continue the discussion, not sidebars

• Check-in with group

• Ensure that senior or vocal person does not shut down

meeting

• Make it fun

The PRR Meeting

32

Candidates for Questionnaire and Issues

• Project objectives

• Project requirements

• Project planning, tracking, and risk management

• Localization

• Training, Knowledge, Tools

• Functional specifications

• Development strategy/process

• Lessons learned

• Communications

???

The PRR Meeting

33

Sample Questionnaire

• Project Objectives• What were the objectives? Were the objectives clearly stated?

• Were the objectives of the project met?

• Did everyone work from the same set of objectives?

• Project Requirements• Was there a Requirements document?

• Were the requirements known, understood and adhered to?

• Did the requirements change? If yes, why and how often?

• Project Planning and Risk Management• Was there a project plan? Was it maintained throughout the

project?

• At the outset, was the schedule realistic?

• Did the project plan change? If so, why and how often?

The PRR Meeting

34

What Does the Sponsor Do After the PPR?

• Collect metrics – Estimated and actual

• Staffing

• Milestones and schedules

• Project cost

• Code size

• Quality (e.g. defects)

• Any other useful metric information

• Collect managerial lessons learned

• Write the PPR document– Use an administrator to help

– Have team review document

• Post or distribute the PPR document

After the PRR Meeting

35

After the PRR Meeting

What Else Does the Sponsor Do After the PPR?

• Follow-up on action items

• Use information in the next or similar projects

• Communicate improvements

36

Outline for PPR Document

• Introduction• Results vs Plan

– Status

– Staffing

– Milestones

– Cost

– Quality

• Lessons Learned– What went well

– What didn’t go well

– Technical lessons

– Managerial lessons

• Recommendations for future projects

After the PRR Meeting

37

Real Life Problems

• The data is not used

• The document is not produced

• People feel that nothing will come from the meeting

• The document takes forever to complete

• Meeting gets out of control

• Key people don’t provide input

Problems

38

What Have We Found

• Consistent issue areas

• Issues moving further downstream in the process

– Requirements and specifications

– Communications

– Quality

– Project planning and tracking

– Process

• People expect follow-through

• A focus on “management” instead of what “I” can do

• Department and organizational project similarities

Discoveries

39

How Have PPRs Helped Us?

• Used the schedule information to plan the next project

• Used actual cost information for project prediction

• Organization metrics

– Time between phases

– Time to market

– Bottleneck areas

• Opportunities to share solutions

Discoveries

40

Summary

• Valuable tool for capturing and learning from past projects

• Valuable tool for predicting future projects

• Good mechanism for project closure

• Great vehicle for tracking organizational information

• Good way for the facilitator to learn about the team

Try it.You’ll like it!!

41

Any Questions??