TPB Internal Review ProcessLSS Project

Vision to Action

2

• Governor McCrory’s Vision of Creating an Efficient and Effective State Government that Focuses on Customer Service Led to;

• Secretary Tata’s Initiative to Implement the Lean Six Sigma Methodology Across DOT by;

• Placing the Governance Office in the Lead Role of Directing Lean Six Sigma Projects, and by

• Training DOT Staff to Lead Process Improvement Projects to Create an Efficient and Effective Agency with a Focus on Customer Service!

Lean Six Sigma – What Is It?

3

Lean Six Sigma Is A Business Improvement Methodology that Integrates Tools and Techniques from Lean Manufacturing and Six Sigma.

Lean

4

• The Lean Methodology Incorporates Process Analysis and Value Stream Analysis to Identify and Eliminate Waste in Processes. The Aim of Lean is:• Eliminating Non-Value Added Steps in the Process• Reducing Process Complexity• Improving Process Flow

• Examples: Defects, Waiting & Unnecessary motion

Six Sigma (6)

5

• Six Sigma is a Methodology and a Management System that Utilizes a Defined Process and Tools to Reduce Process Variation and Eliminate Defects.

• It Is a Top-Down Approach• Critical Improvement Efforts are Aligned with Key

Business Goals• Problem Solving is Data-Driven

Lean and 6 Together – A Powerful Combination for Improvement

6

• Lean and Six Sigma Are Complementary

• By Reducing Waste AND Reducing Variation We Can Achieve Better Quality and Greater Speed, Contributing to Lower Cost

DMAIC

7

5 Distinct Phases

• D – Define the Project, the Process and the Opportunity for Improvement

• M – Measure the Process to Understand Process Performance

• A – Analyze the Process to Identify Root Causes of Current Performance

• I – Improve the Process to Achieve Desired Gains

• C – Control the Process to Maintain the Gains

8

Project Participants

Champion / Sponsor – Susan Pullium, Director of Strategic

Planning

Project Leads – Earlene Thomas, TPBDoug Cox, Governance Office

Team Members – Brendan Merithew, TPB Suzette Morales, TPB Daniel Sellers, TPB Hemal Shah, TPB

9

Process Map

10

The Project Charter

Problem Statement/Mission:

• TPB's internal review process for approval of draft CTP maps, problem statements and reports is required to be completed within six weeks.

• During the previous twelve months the target was met only 21% of the time.

• Missing these target requirements results in delays in product delivery, missed PDA milestones, and poor customer service.

11

Voice of the Customer

Who is the customer?

TPB staff:

Project engineers (~30)

Planning group supervisors (6)

Unit heads (2)

Branch manager (1)

12

Voice of the Customer

What did we learn from them?

Project Engineers – Experiences:• Project Engineers do not always receive needed feedback• Comments/feedback from supervisors can be inconsistent• Expectations from supervisors are not always clear

Project engineers:• Offer lowest ratings in the consistency of feedback received

13

Voice of the CustomerWhat did we learn from them? (continued)

Supervisors – Experiences:• Reports are frequently not up to standards (spelling, formatting)• Information and data in report is not always accurate/justified• Submission needs to follow prescribed preferences/guidelines

Supervisors:• Offer lowest ratings regarding completeness and consistency of

products submitted for review• 83% of supervisors say the number of errors in a submitted

product was substantial

14

Measurement System AnalysisCycle Time MSAStart and end data collected from electronic records maintained by planning group supervisors, unit heads, and the branch manager of CTPs submitted for review.• End dates for the review/approval process maintained by the branch manager.• Start dates for the review/approval process maintained by the planning group

supervisors are not as reliable.• Some CTPs did not have a start date.• Some CTPs had a start date that could not be verified.• Only those CTPs with a reliable start date were included in the study.

Number of IterationsData collected from electronic version of CTPs. Reviews conducted and maintained at supervisory level indicates for each CTP:

• Number of iterations in review process• Total number of errors (formatting versus others)

15

Data AnalysisDescriptive Statistics of Time in Process

Count (number of products) 37Mean (number of days) 113Stdev 81.81Range 285Minimum 825th Percentile (Q1) 45.550th Percentile (Median) 7775th Percentile (Q3) 182Maximum 293

• Only 8 of 37 products met the target of six weeks for review/approval (42 days).• The average number of days required for review/approval is 113 days.

Target Cycle Time (42 days)

Average Cycle Time (113 days)

1 Sample t-Test

Test InformationH0: Mean (Mu) = 42 daysHa: Mean (Mu) Greater Than 42 days

Count 37Mean 113p-value (1-sided) 0.0000LC (1-sided, 95%) 90.291

• Reject the null hypothesis (p < .05)

16

Data AnalysisComparison of Planning Groups/Units in Meeting Target

• Data suggest that delays are not related to any particular group or unit• Data suggest that product type is not related to delays in the review/approval

process.• Reducing the number of errors in a product will require fewer iterations, thus

reducing the number of days in the review/approval process.

Review of Process Flow Map• A review of the process flow map shows that each time a supervisor/unit

head/branch manager does not approve the product it is returned to the project engineer for rework. These defects in the process are non-value added steps that contribute to the review/approval process taking longer than the target of 42 days.

17

Root CausesTraining/Experience Lack of experience/competency Competency level using GIS Training occurs at the beginning and not along the way Lack of technical training for the development of problem statements Supervisors who have never done a CTP lack project engineer point of view Relatively few reviews per year, leading to lack of consistency Attempting to develop a product from a sample or previous product Conflicting/unclear goals with problem statements – major vs. minor Strategic Highway Corridor (SHC)

Inconsistency of Supervisory Levels Varying supervisory style/direction Supervisors and unit heads contradict each other Inconsistent/incompatible directions at management levels Supervisor checklist does not match comments given Lack of implicit rules

Staffing Turnover Staff on leave Personality issues

Delays Scheduling issues – day-to-day delays and/or long term absences Knowing reports/maps are not immediately needed, resulting in not pressing the issue Excessive queues of reports without review Lag time between submission, review, revisions

Performance Performance issues Product not thoroughly reviewed prior to submittal Accuracy of the product/data/analysis

Rework Simple comment/point that causes considerable rework Having to reformat document when changes are made Constant changes Have to redo some parts that were once just part of the appendix Finding new problems in second and third review when they were previously not a

problem

Communication Communication of comments Standards/procedures/format have not been forwarded Poor management of changes in standards Not knowing proper format to submit product (hard or soft) Not sure if project engineer is grandfathered for process update When does the clock start – alterations in the process impact the timeline Finding reviewers in their office to consult or challenge comments

Workload Workload requires that revisions/reviews be reprioritized Managing multiple incoming reviews Competing priorities One person is not enough Travel Is six weeks for review long enough

Size of Reports Reports are too lengthy, takes too much time to review CTPs have a large number of problem statements, maps, inserts, appendices

Printer and Software Printing/assembling paper reports Printer not working properly Loss of work for unknown reasons Compatibility of software and printer

Missing Data/Information Missing information/request for additional information Lack of data Transferring local knowledge to supervisors/unit heads

All Others During review, study areas require change Process improvement initiatives – implementation Documenting exceptions or unique features with CTP Inconsistency among agencies and their knowledge base School delays and closings

18

Pilot the SolutionDescription of the Solution:• A peer review team that will review CTP products prior to being submitted

will be piloted.• The purpose of the pilot is to determine if the proposed solution will result in

shortening the review/approval cycle time by reducing the number of errors contained in the product and, thus, the number of iterations.

Method:• Three products will be reviewed independently by the peer review team and

the group supervisor.• The peer review team and the group supervisor concurrently review the

product and quantifies the number of errors detected.• The results of the three product reviews will be compared to determine if the

peer review detects an equal or greater number of errors than the group supervisors.

19

Effectiveness of the Pilot

Pilot Demonstrated:• Peer Review detected 50 – 110% more errors than during the supervisory level

review

Inferences:• Having products reviewed prior to submittal for supervisory review/approval

allows for errors to be detected and corrected in advance• Detecting common errors prior to supervisor review allows supervisors to focus

more on specific details instead of general structure• Product reviews can potentially be completed within the target time of 42 days

20

Overview of what to expect:High-level review of CTP Products

Looking for basic errors – NOT rewriting, researching or editing. Don’t need to be an expert on the CTP area.

Meet as a group to briefly discuss commentsGo over merged comments. Opportunity to discuss CTP standards and style.

Mandatory participation, peer comments do NOT affect performance review

Project Engineer makes edits accordinglyPE edits document on own time. It is up to the PE whether or not to implement comments.

Not being evaluated on quantity or type of comments. This is a “free” quality control check prior to submitting project to supervisor.

21

Implement the Solution – How make it Work

• Starting July 1, 2015 Peer Review of all CTP Products is

mandatory to every engineer in the Branch, it is aligned with

the start of the new NCVIP cycle

• Products for review: CTP Maps, PS and CTP Report

• Procedure for the Peer Review process published

• CTP tracking tool, schedules and milestones are updated

accordingly

• Training on process

22

Peer Review Team• Mandatory 4 person teams assignments –

tracked by Staff Engineer

• Branch wide balanced participation –

at least one representative from each

(not from the same group as PE) :

Eastern planning

Western planning

Non planning (forecaster or modeler or systems planner)

• Selection based on the Random Numbers

• Mixed levels teams: Contributing, Journey, Advanced

• Tracking posted on the S drive S:\Shared\TPB Temporary\Peer Review

23

Time Requirements and Scheduling

• Total estimated time at about 8 man hours

• Product must be submitted to Staff Engineer by noon on Wednesday

• Team assigned and products emailed to the team by EOB Friday

• Peer review team provides comments by EOB Friday of the assigned week.

• During the assigned review week PE schedules in person meeting with the

team to review comments.

• All conflicts should be brought to the attention

of the Staff Engineer immediately

24

Peer Review Procedurehttps://connect.ncdot.gov/projects/planning/TransPlanManuals/CTP_PeerReview.pdf

• PE submits products electronically (MS Word format for text and PDF for maps) to

TPB Staff Engineer prior to NOON Wednesday each week.

• TPB Staff Engineer selects/assigns products to the Peer Review team and uploads

all the products to the TPB Teamsite Docs library on Inside NCDOT

• TPB Staff Engineer notifies PE, Team Members, and all the supervisors about the

assignment same week by EOB Friday

• Team completes individual reviews on the Teamsite by EOB Friday.

• During the assigned review week PE schedules in person meeting with the team to

review comments.

• PE downloads final document from the Teamsite to his/her own computer

25

Peer Review Procedure (continued)https://connect.ncdot.gov/projects/planning/TransPlanManuals/CTP_PeerReview.pdf

• Comments should be made using the Comment feature for text in MS Word and

Sticky Notes feature in Acrobat for pdf maps

• During the team meeting with PE on the scheduled date, peer review

comments are reviewed/discussed.

26

Benefits

• Improve internal review times• Improve customer service• Reduce labor costs• Improve consistency of reports and reviews• Meet standards before supervisory submission• Learn from peers• Allow other groups to have a better understanding of

CTPs

27

Review Expectations/Criteria• Review maps for completeness/display• Maps and Problem Statements (PS) Match• PE follows PS Standards• PE follows Quick Tips• PE uses correct grammar • References in report match actual figures, tables, etc.• Learning opportunity for PE and Review Team

28

What NOT to Expect

• Performance Evaluation• Thorough review of technical analysis• Detailed Local Knowledge• Validation of Information• Catch Everything• Peer review to replace supervisory review• Approval of Maps and Reports by Peer

Review team• Full consensus on all comments

29

Example of Maps Peer Review

Paper Copies: • Step 1: Print maps and write comments on the maps• Step 2: Scan/Email maps with comments using copier to send to PE

ORAdobe Reader:• Step 1: Open the map in Adobe Reader• Step 2: Click on Comment tool to select “Add Sticky Note” • Step 4: Click on the map and write comment on the sticky note.• Step 5: Save and email map with comments to PE

30

Step: 1Step: 2

Step: 3

Step: 4

31

• Step 1: Open Internet Explorer (Recommended)• Step 2: Go to Transportation Planning Branch Team site on Inside NCDOThttps://inside.ncdot.gov/Teams/transportationplanningbranch/pages/default.aspx• Step 3: Open the file listed for Peer Review in Edit mode.• Step 4: Suggest edits and/or make comments.

Example of Report / PS Peer Review

Step: 3

32

• Multiple people can edit the word document at the same time on the TPB Team site, due to the SharePoint capabilities.

• All the changes/ edits / comments made will be automatically saved in real time.

• To see the changes made to the word document Open word document > Go to Review tab. Click on Show Markup Choose Reviewers to display a list with each Reviewer's name

Utilizing Share Point

33

34

Conclusions

• Opportunity to catch worst mistakes upfront

• Bring uniformity to branch CTP products

• Learn from each other

• Understand what supervisors go through

Questions?