identify and analyze possible causes (x’s) identify and

Identify and analyze possible causes (X’s) for the undesirable output

Identify and understand which of the possible causes (X’s) are the biggest contributors to the undesirable output

Identify which causes (X’s) are within the team’s control and those outside their control

Identify methods to verify the suspected big causes (X’s)

2

Identify what data should be collected to validate the suspected big causes (X’s)

Identify and perform appropriate statistical tests to confirm suspected big causes (X’s)

Determine team commitment to improvement targets for the big causes (X’s)

Review and amend Cost of Poor Quality (COPQ) estimates

Develop class project Analyze phase presentation

3

Ishikawa (Fishbone Diagrams)

5 Why’s

Failure Mode and Effects Analysis (FMEA)

Charts / Plots (Box & Whisker)

Correlation and Regression Analysis (SLR)

Hypothesis Testing

4

Process Flow Analysis

Brainstorming

Pareto Charts

Check Sheets

Capability Analysis

Control Charts

Design of Experiments

Gap Analysis

Waste Analysis

Cost of Poor Quality

5

Phase Objectives Key Activities Possible Tools and Techniques Key Deliverables Document the problem statement and establish the charter. Demonstrate alignment with the Business metrics and strategies. Determine Customer requirements and performance standards.

▪ Select Team with Champion

▪ Develop problem statement

▪ Develop Charter ▪ Create SIPOC ▪ Address gap between VOC

and process ▪ Estimate financial benefits

▪ Problem Statements ▪ Project Charter ▪ SIPOC map ▪ COPQ or CODND ▪ Communication Plan

Develop a reliable and valid measurement system of the business process to effectively evaluate the success of meeting customer requirements.

▪ Obtain Customer requirements

▪ Create overall project plan ▪ Develop measurement

plan & compile project metrics

▪ Determine defect tracking requirements

▪ Assess baseline performance-estimate process capability

▪ Measurement Systems Analysis

▪ Process Description ▪ Project Plan & Timeline ▪ Metrics and collection plan ▪ Baseline Performance results ▪ Process capability analysis ▪ Lean Tools Assessment ▪ Measurement Systems

Analysis ▪ Process model – ‘as is’

Utilization of data techniques to gain insight into process. Divide data into groups based on key characteristics and assess the root causes of errors and poor performance. Determine where to focus efforts for improvement.

▪ Statistical tests / tools ▪ FMEA ▪ Pareto chart ▪ Correlation/Regression ▪ Fishbone Diagram ▪ Box plot ▪ Hypothesis Testing

▪ Describe findings – identify potential root causes RCA

▪ Validate findings

▪ Data relationships ▪ Validated Key Input Variables

(KPIVs) & Key Output Variables (KPOVs)

▪ Prioritize sources of variation ▪ root causes

▪ Identify & communicate potential improvements

▪ Summarize benefits & annualized financial benefits

Identify key change opportunities and proactively test for optimization. Develop implementation and communication plan including a change management approach to assist the organization in adaptation of the improvements.

▪ Design of Experiments – describe purpose & build test/ analysis strategy

▪ Evaluate and Confirm results

▪ Analyze KPIVs ▪ Create action plan for

implementation including change management and communication needs

▪ Buy-in assessment

▪ Quantified relationship between key

input and key output variables ▪ Defined process improvements

including impacts and benefits ▪ Implementation Plan ▪ Process model – ‘Should be’ ▪ Impacted Employees are Trained

Definition of optimal process settings and conditions with specified metrics. Implementation of improvements with a control plan to assess & maintain gains.

▪ Implement improvements ▪ Evaluate results ▪ Integrate & manage

improvements in work processes

▪ Complete closure activities

▪ Document process change ▪ Control plan ▪ Determine new process capability ▪ Leverage opportunities for replication ▪ Communicate results ▪ Financial audit ▪ Hand-off to process owner

1.0 Define

Opportunity

2.0 Measure

performance

3.0 Analyze

Opportunity

4.0 Improve

Performance

5.0 Control

Performance

6

Six Sigma Process Improvement Road Map

1

Migration e-Pro Process ImprovementProject Charter

Project Description Error corrections and clarification of benefits are generatingrework throughout the migration and case installationprocesses, accounting for 20% of the total number of e-Prochange transactions. It is estimated that the volume of errorand rework will grow proportionally as the number ofaccounts migrating by 1/1/2004 increases, driving aproportionate increase in cost and potentially dissatisfyingcustomers.

Start Date April 1, 2003

Completion DateScheduled to be completed by September 5, 2003

Baseline Metrics For 1/1/03 migrated accounts:National Accounts- Average number of change transactions: 14.3, of which

2.9 are due to error and rework- Average hours of rework: 309 hoursRegional Accounts- Average number of change transactions: 8.0, of which

1.6 are due to error and rework- Average hours of rework: 137 hours

Primary Metrics 1. Total e-Pro change transactions2. Percentage of change transactions due to error and

benefits clarification3. Average rework hours per error and benefits clarification

Secondary Metrics none

Goal Reduce error and rework in the migration process by 50%starting with 1/1/04 migrating accounts

Customer Customer migration survey results

Financial CODND (Cost of Doing Nothing Differently)4

th Qtr 2003: $500K

Year 2004: $2.5M

Ben

efits

Internal Productivity Estimated cycle time reduction of 18,868 hours (assuming195 accounts migrating 1/1/04).

Define April 1 – April 21, 2003

Plan Projects & Metrics April 14 – April 18, 2003

Baseline Project April 21 – May 2

Consider Lean Tools May12 – May 16, 2003

MSA May 19 – June 2, 2003

Wisdom of the Org. June 2 – June 6, 2003

Passive Analysis June 9 – June 20, 2003

Proactive Testing June 23 – August 4, 2003

Ph

as

e M

ilesto

nes

Control August 4 – September 5, 2003

SUPPLIER INPUT PROCESS OUTPUT CUSTOMER

Sales

Client / Policy HolderHR BenefitsCoordinator

Client Consultant

Third Party BenefitsVendor

Member

GO Decision

Policy Renewal Date

Summary of Benefits

AdministrativeRequirements

AccountOrganizational

Structure

Detail Benefits

Account DataLoaded in System

Member andDependentEligibilityInformationLoaded in System

Member ID Card

Client / PolicyHolder

Third Party BenefitsVendor

Member andDependent

Providers

Claim

Call

1. Conduct migration analysis

2. Complete account profile

3. Load account structure in system

4. Set up and validate account benefits in system

5. Produce account eligibility record

6. Load account data in product claim engines

0Subgroup 10 20 30 40

0

10

20

30

Ind

ivid

ua

l V

alu

e

Mean=10.98

UCL=26.81

LCL=-4.854

0

10

20

Mo

vin

g R

an

ge

1

R=5.952

UCL=19.45

LCL=0

Total e-Pro Change Transactions by Account from Sep 2002 thru Mar 2003Conduct

Analysis

Create Implementation

Guide

Expert Team

Meeting

Draft

EPRO

Draft

e-

PRO

Release e-PRO

Record

e-

PRO

Impl.

Guid

e

Update

EPRO

OK For

Release

to Vendor

Track

Systems

Loads

IMPLEMENTATION

ERW

From

Eligibilit

y

GO

Decisio

n

SALE

S

Set Up Client ID in

End State

Structure

Request

Codes

STRUCTURE

Complete

Structure

Inspection/Verify

with e-PRO

Get

Underwriting

Approval

Yes

No OK? Yes

No

Go back to

Rates

Structur

e in

CDB

No

Yes

Release

ATC

To Vendor

OK?

Review

Draft e-

PRORequest/Receive

Codes

Enrollmen

t File

CLIENT /CUSTOMERClient

Input

To Sales

Run Legislative

Tool

Check vs. e-

PRO

e-PRO

Redo?

Yes

No

YesOK?

Review

Draft

e-PROCreate

Codes

Legislative Tool

Review

BPC &

Class

Codes

To

Structure

BENEFITS

TS

ID Claim

Scenarios

Load Data into

Downstream

Systems

OK?Yes

No

Data

Engines

Loaded

(e.g ATC,

DocGen,

etc)

Test Scenarios

Check vs. e-

PRO

OK?No

Yes

Fix Claim

Errors

No

No

VOB

Yes

EPRO

Rework

CDB

From

Vendor

Member

cancelle

d

in

Legacy

Reformat

Client

Eligibility DataReview

Draft

EPRO

ELIGIBILITY

Receive Enrollment

Data

Match &

Merge

Load data in

CEO

Are

errors

resolve

d?

Fix Errors

YesNo Cancel

Member in

Legacy

Create

ERW

ERW

Eligibility

In CED

VENDO

R(ID

CARDS)

ID

Card

s

From

Benefit

s

Get

Underwriting

Approval

ERW To

Implementatio

n

ERW To

Implementatio

n

Create Client IDClient

ID

To

Structure,

Benefits,

and

Eligibility

Migration

Structure

Mapping Job

Aid

GO

Decision

To

Structure,

Benefits,

and

Eligibility

e-PRO

Rework

EPRO

Rework?

SMT linking

legacy

structure to

end state

To

Eligibility

CAIP

Processes shaded in green are specific to

migration

Processes out of scope, but critical to

Employer Services

Employer Services functional areas

OUT OF SCOPE

PROCESS STEPS

Production

Migration

Support

Cancel

Legacy

Structure

Elig.

Rework

e-Pro

Rework

Rework Loops highlighted in Red

10

100

50

0

Contracting

T4

-T1

PMHS

Overpayment

Process Data / Materials

PeopleTechnology

Ÿ Auth / Referral Info missing/incomplete/incorrect

Ÿ OI Info missing/incomplete/incorrect

Ÿ Member Eligibility Info missing/incomplete/

incorrect

Ÿ Benefit Info missing/incomplete/incorrect

Ÿ Provider Fee Schedule Info missing/incomplete/

incorrect

Ÿ Provider/Vendor TIN/SSN Info missing/

incomplete/incorrect

Ÿ Additional Information Necessary to Process

Claim

Ÿ Transaction/Codeset data excluded at gateway

Ÿ Standard Operating Procedures (SOPs)

Ÿ Claim Audit Process >$5K

Ÿ Second/Third Party Internal Review

(Medical Management, Claim Benefit

Build)

Ÿ iTrack - drives usage of paper reports to

sort older claims

Ÿ Skill Level of Processor

Ÿ Accessiblity of Site Coach/Training Staff

Ÿ Aggressive Productivity goals conflict with low

quality requirements

Ÿ Rushed Training Schedule

Ÿ Lack of up-training / reinforcement training

Ÿ Best Practice / Skill Training not conducted

Ÿ OJT training on SOP usage

Ÿ System Error During Processing

Ÿ Data Fallout

Ÿ Aurhorization Mis-Match

Ÿ System Restrictions - LPI Manual Calc

Ÿ Data Set Up Issues (eligibility, provider, benefits)

Ÿ Timeliness of Batch Processing

Ÿ Bank Acct Set-Up Delays

Ÿ Customer Touchpoints Delays

Ÿ Inappropriate assignment or missing hold codes

Ÿ Provider Mis-Match

Ÿ Transaction Limitations on data collected at

gateway

Manual Adjudication &

PMHS Provider Selection

- Manual

End

Manually check

provider/ vendor

on claim system

Check provider

data and claim

data against iView

image

Mismatch?Manually try to find

correct data

Found data

Correct data &

verify COB

Service request to

appropriate area

YES

NO

YES

iTrack

Verify in claim

screen and follow

COB Checklist

Attempt to

adjudicate claim

NO

Claim processed

Hold codes that

require further

research

NO

Re-open the claim

YES

Process will

depend on Hold

Code & SOP/ Job

Aid

CIRF

Attempt to resolve

all Hold Codes at a

line level

Resolve service

requests

Adjudicate claim

(manual or

systematic)

End

ID Task Name

1 DEFINE PHASE

12 MEASURE PHASE

13 Plan Project and Metrics

22 Baseline the Project

23 Select KPOV metric to track process output

24 Estimate process capability/performance at the 30,000-foot-level

25 Categorical failures

26 Create pareto chart

27 Rescope project to a large Pareto category

28 Repeat Baseline the Project steps 23 through 27

29 Non categorical failures

30 Revise estimate for COPQ/CODND

31 Project status update w ith executive sponsor

32 Consider Lean Tools

39 Conduct Measurement Systems Analysis (MSA)

40 Ensure data integrity

41 Perform Gauge R&R

42 Improve gauge


44 Wisdom of the Organization

55 ANALYZE PHASE

56 Use visualization of data techniques to gain insight to processes

57 Conduct inferential statistical tests and confidence interval calculations on individual KPOVs

58 Conduct appropriate sample size calculations

59 Conduct hypothesis tests

60 Describe statistical f indings to others using visualization of data techniques

61 Implement agreed-to process improvement findings


63 IMPROVE PHASE

65 d13 d

18 d

18 d

18 d

20 d

20 d

20 d

23 d

23 d

23 d

27 d

76 d33 d

69 d38 d

38 d

40 d

43 d

69 d48 d

61 d48 d

53 d

60 d53 d

60 d53 d

60 d53 d

60 d53 d

53 d

58 d

62 d

63 d

02 09 16 23 30 06 13 20 27 04 11 18 25 01 08 15 22 29 06 13 20 27 03 10 17 24 31 07 14 21 28 05

March April May June July August September Octob

PMHS

# of Audits

7,321

04/05/2003

# $'s

Under 300 4% Under 1,030,680$

Over 676 9% Over 2,303,562$

No $ Error 914 12% No $ Error -$

No Error 5,431 74% No Error -$

7,321 3,334,243$

A)SEVERITY B)OCCURRENCE

Probability

C)DETECTION

Probability

RISK

PRIORITY

NUMBER ACTION TO IMPROVE

Rate 1-10 Rate 1-10 Rate 1-10 RPN

10=Most

Severe

10=Highest

Probability

10=Lowest

Probability AxBxC A B C

Provider Mis-Match 10 8 9 720

Provider Data Incorrect/Incomplete 9 8 9 648

Data Fallout 9 6 10 540

Data Set Up Issues 9 6 10 540

Provider Fee Schedule Unclear 9 6 9 486

OI Information Needed 9 6 6 324

System Restrictions 6 6 9 324

Hold Codes 9 3 10 270

FAILURE MODE

Process Name: PMHS Claim Processing

Date: 6/30/2003 Revision Level: 3

REVISED VALUES

0Subgroup 10 20 30 40

0

10

20

30

In

div

idu

al

Valu

e

Mean=10.98

UCL=26.81

LCL=-4.854

0

10

20

Mo

vin

g R

an

ge

1

R=5.952

UCL=19.45

LCL=0

Total e-Pro Change Transactions by Account from Sep 2002 thru Mar 2003

Gage R&R http://www.aiag.org/ Part Number http://www.qimacros.com/free-lean-six-sigma-tips/aiag-msa-gage-r&r.html

Average & Range Method 1 2 3 4 5 6 7 8 9 10 Sum

Appraiser 1 Trial 1 0.65 1 3.250

Enter your data here-> Trial2 0.6 1

Trial3

Trial4

Trial 5

Total 1.25 2

Average-Appraiser 10.625 1 #N/A #N/A #N/A #N/A #N/A #N/A #N/A #N/A

Range1 0.05 0 #N/A #N/A #N/A #N/A #N/A #N/A #N/A #N/A

Appraiser 2 Trial 1 0.55 1.05 3.100

Enter your data here-> Trial2 0.55 0.95

Trial3

Trial4

Trial 5

Total 1.1 2

Average-Appraiser 20.55 1 #N/A #N/A #N/A #N/A #N/A #N/A #N/A #N/A

Range2 0 0.1 #N/A #N/A #N/A #N/A #N/A #N/A #N/A #N/A

Appraiser Trial 1

Enter your data here-> Trial2

Trial3

Trial4

Trial 5

Total

Average-Appraiser 3#N/A #N/A #N/A #N/A #N/A #N/A #N/A #N/A #N/A #N/A

Range3 #N/A #N/A #N/A #N/A #N/A #N/A #N/A #N/A #N/A #N/A

EV (Equipment Variation)0.0332 Equipment Variation (EV)

%EV 11.3% 39.9% # Parts #Trials #Ops % of Total Variation (TV)

AV: (Appraiser Variation)0.02066 2 2 2 Appraiser Variation(AV)

%AV 7.0% 24.8% % of Total Variation (TV)

R&R (Gage Capability) 0.0391 Repeatability and Reproducibility (R&R)

%R&R 13.3% 47.0% NDC 11 % of Total Variation (TV)

PV (Part Variation) 0.2917 Part Variation (PV)

%PV 99.1% 350% % of Total Variation (TV)

Skewness0.41Stdev1.670.20Max6.20

There are two different sets of tools for determining the Root Cause of issues in the process◦ Subjective

◦ Analytic

We will cover the most commonly used tools in this module

8

What is a ‘Root Cause Analysis’?

◦ Reactive assessment of basic or contributing causal factors associated with a specific event

In English – what was the real “cause” of the issue!

◦ Analysis focused primarily on system and process issues rather than assigning individual responsibility

9

A method used to help drill down into the process steps to determine the basic causal factors associated with each failure mode

Allows the group to obtain root cause information about an event

Uncovering root causes is critical to process improvement.

If left uncovered the team is simply “band-aiding” a problem temporarily

10

Event Occurs

Take Immediate

Action

Enter Event into

Information System

RCA Required?

Complete Root Cause Analysis

Implement Corrective

Action

Evaluate Corrective

ActionAgainst Goals

Subjective Tools

Analytic Tools

Root Cause Statements

11

Often used 1st on the process

Often used to verify subjective tests

Subjective (Soft) Tools◦ Flow Analysis (M)

◦ Ishikawa (Fishbone)

◦ 5 Whys

◦ FMEA

◦ Graphical Analysis

◦ Brainstorming Process (I)

Analytic Tools◦ Pareto (M)

◦ Checksheets (M)

◦ Capability Analysis (M)

◦ Control Charts (M)

◦ Regression Analysis

◦ Analytical Tests (Hypothesis Testing)

◦ Design of Experiments –DOE (I)

12(M) = covered in the Measure Phase; (I) = covered in the Improve Phase

• Also known as Ishikawa fishbone diagram• Visual representation of known causes to a particular

effect

• Allows the team to drill down in a systematic way to identify major contributing KPIV’s

13

An Ishikawa Diagram (a.k.a fishbone diagram)

can be used to map the process input

variables (PIVs) that affect each KPOV.

14

HEALTHCARE PATIENT TREATMENT FLOW

Step 1 Step 2 Step 3Patient

Outcome

Output of

Treatment

Step

Equipment

Materials Environment

People

Methods

Information

HEALTHCARE PATIENT PROCESS VARIATION

Var(Process) = Var(Step 1) + Var(Step 2) + Var (Step 3) + . . .

Var( Treatment Step) =

Var(Methods) + Var(Materials) + Var(Environment)

+ Var(People) + Var(Equipment) + Var(Information)

Inspection Time appears

To be Excessive

KPIVs

KPOV

15

Analyze 3 – ISHIKAWA diagram template

As a class, construct a Fishbone Diagram for the Class Scenario◦ You can use sticky notes or use a spreadsheet to

list the factors in the columns

◦ Start by determining the outcome

◦ Fill in the “bones”

Write down all the factors

Don’t debate their relative merits; the purpose is to understand the possible inputs

The class has 5-10 minutes for this exercise

16

For each step in the process ask:◦ What problems occurred during this step?

◦ Why did these problems occur?

If the answer from the first question does not provide the root cause, keep asking “why?” until the root cause is reached.

17

Patient leavesHospital bed to go to

bathroom

Patient slips

and falls

Patient found on floor with broken

hip

Why did the patient leave the bed? To go to the bathroom

Why did the patient go to the bathroom unattended? Patient was not deemed a fall risk, normal precautions taken.

Why was the patient not deemed a fall risk? Fall assessment score was well below threshold.

Why was the patient fall risk below threshold?Fall assessment tool does not address specific medications taken.

18

19

Patient leavesHospital bed to go to

bathroom

Patient slips

and falls

Patient found on floor with broken hip

Why did the patient slip and fall? Patient lost balance on the way to rest room.

Why did the patient lose his balance?Patient was not wearing non-slip socks

Why was the patient not wearing non slip socks?Patient recently admitted, socks not available on floor.

Why were there no non-slip socks on the floor?Par-level was too low, restocking was in progress.

Why did the restocking take so long?Weekend, night shift, low staff coverage

When I fill my gas tank it overflows. Can I determine the root cause?

At work we run out of material. Can I determine the root cause?

20

◦ Used to identify all possible failure modes and their effects on a system

◦ Used to identify critical parameters

◦ An excellent tool for supporting a company’s commitment to continually improve products and services wherever possible

◦ Can focus on a process or the design of a new product

21

Think of this as a priority list;

NOT of the things that can go wrong… BUT the things that have to go right.

2 Types of FMEA• DFMEA – Design FMEA

• PFMEA – Process FMEA

22

Improved product functionality & robustness

Reduced Warranty costs

Reduced day-to-day operations issues

Improved safety of products & implementation process

Reduced business process problems

23

Know it is a living document and needs to be reviewed periodically

Conduct early in an improvement to:◦ Design out failure modes by identifying/removing

root causes

◦ Reduce seriousness of failure if elimination is not possible

◦ Reduce the occurrence of failures

◦ Improve detection of failures

24

Note an input to a design or process ( e.g. process step, KPIV, Cause & effect matrix)

List 2 or 3 ways the input/function can go wrong (a failure)

List at least one effect for each potential failure mode

For each failure mode, list 1 or more causes of input going wrong

For each cause list at least 1 method of preventing or detecting the failure

Enter SOD values

You can use the template DOE Gage R&R FMEA >

Failure Mode Effects Analysis

◦ Tab PFMEA (A)

25

Process Operation:◦ Process step under investigation

Process Failure:◦ Way the process could fail to meet the customers

requirements. Every process parameter failure should be taken into account even it is controlled

26

Process

Operation

Process

FailureEffect SEV Cause OCC Controls DET RPN

Actions

Taken

Effect:◦ Effect of the process failure on the product, process

parameter, or customer Severity (SEV)◦ Rank on a scale of 1 to 10. The highest #10 associated with

a safety concern and the lowest #1 associated with a non-concern.

27

Process

Operation

Process


Actions

Taken

Rating Description Definition (Severity of Effect)

10 Dangerously high Failure could injure the customer or an employee.

9 Extremely high Failure would create noncompliance with federal regulations.

8 Very high Failure renders the unit inoperable or unfit for use.

7 High Failure causes a high degree of customer dissatisfaction.

6 Moderate Failure results in a subsystem or partial malfunction of the product.

5 Low Failure creates enough of a performance loss to cause the customer to complain.

4 Very Low Failure can be overcome with modifications to the customer’s process or product,

but there is minor performance loss.

3 Minor Failure would create a minor nuisance to the customer, but the customer can

overcome it without performance loss.

2 Very Minor Failure may not be readily apparent to the customer, but would have minor effects

on the customer’s process or product.

1 None Failure would not be noticeable to the customer and would not affect the customer’s

process or product.

28

Caution! Severity ranking should NOT be considered low just because its occurrence is low, or because its detection is very effective.

Note: A reduction in SEVERITY rank is normally achieved through a design change to the system/sub-system that uses the device.

29

Cause:◦ How could the failure occur? Is there something that could

be controlled?

Occurrence (OCC):◦ Rank on a scale of 1 to 10, on the basis of the likelihood

that the process failure will occur. Rank of 10 meaning the failure is sure to occur and 1 meaning the failure is unlikely to occur.

30

Process

Operation

Process


Actions

Taken

Rating Description Potential Failure Rate

10 Very High: Failure is

almost inevitable.

More than one occurrence per day or a probability of more than three occurrences

in 10 events (Cpk < 0.33).

9 High: Failures occur

almost as often as

not.

One occurrence every three to four days or a probability of three occurrences in 10

events (Cpk ≈ 0.33).

8 High: Repeated failures. One occurrence per week or a probability of 5 occurrences in 100 events (Cpk ≈

0.67).

7 High: Failures occur often. One occurrence every month or one occurrence in 100 events (Cpk ≈ 0.83).

6 Moderately High:

Frequent failures.

One occurrence every three months or three occurrences in 1,000 events (Cpk ≈

1.00).

5 Moderate: Occasional

failures.

One occurrence every six months to one year or five occurrences in 10,000 events

(Cpk ≈ 1.17).

4 Moderately Low:

Infrequent failures.

One occurrence per year or six occurrences in 100,000 events (Cpk ≈ 1.33).

3 Low: Relatively few

failures.

One occurrence every one to three years or six occurrences in ten million events

(Cpk ≈ 1.67).

2 Low: Failures are few and

far between.

One occurrence every three to five years or 2 occurrences in one billion events

(Cpk ≈ 2.00).

1 Remote: Failure is

unlikely.

One occurrence in greater than five years or less than two occurrences in one

billion events (Cpk > 2.00).

31

Controls:◦ What are the controls that are currently in existence to prevent

process failure from occurring OR to detect the effect of failures.

Detectability (DET):◦ Rank on a scale of 1 to 10, based on the probability that the

process controls will detect the process failure (prevention) or the effect of the process failure (detection).

◦ Rank of 10 indicates that there is absolute certainty of non-detection and 1 means the control is certain to detect the failure.

32

Process

Operation

Process


Actions

Taken

Rating Description Definition

10 Absolute Uncertain No inspection or the defect caused by failure is not detectable.

9 Very Remote Product is sampled, inspected, and released based on Acceptable Quality Level

(AQL) sampling plans.

8 Remote Product is accepted based on ‘no defectives’ in a sample.

7 Very Low Product is 100% manually inspected.

6 Low Product is 100% manually inspected using go/no-go or other mistake-proofing

gauges.

5 Moderate Some Statistical Process Control (SPC) is used in process and product is final

inspected off-line.

4 Moderately High SPC is used and there is immediate reaction to out-of-control conditions.

3 High An effective SPC program is in place with process capabilities (Cpk) greater than

1.33.

2 Very High All product is 100% automatically inspected.

1 Almost Certain The defect is obvious or there is 100% automatic inspection with regular

calibration and preventive maintenance of the inspection equipment.

33

Risk Priority Number (RPN):◦ Quantifies the risk associated with a given process failure

mode.

RPN = Severity (S) x Occurrence (O) x Detection (D)

RPN ranks between 1 and 1000

Caution! Even if the RPN is low, a severity rating of 10 needs to be addressed.

Action:◦ Activity that needs to be initiated due to high risks

identified by the RPN rating. Although there is no rule for a threshold, typically above 125 is considered an actionable level.

34

Investigate the process and data to determine the biggest contributors to the undesirable output (Y = f(X))

Find patterns and data to support observations

Separate the vital few from the trivial many KPIV’s

Prove the major contributors with data

36

Graphical Analysis reveals apparent signs of process differences leading to potential solutions◦ Example: The Box Plot will show the differences

in variation for multiple groups for data

Statistical Analysis proves statistical differences which can be exploited for finding solutions; graphical analysis is used as a prelude to statistical analysis

3838

You have already seen:◦ Pareto Diagram◦ Run Charts◦ Histograms◦ Control (SPC) Charts

We will introduce:◦ The Box and Whisker Plot

Other tools on QI Macros include:Dot PlotsScatter PlotsMulti-Vari ChartsValues Plot

39

40

-0.006771

0.0132292

0.0332292

0.0532292

0.0732292

0.0932292

0.1132292

3/21/2008 3/22/2008 3/23/2008 3/24/2008 3/25/2008 3/26/2008 3/27/2008 3/28/2008 3/29/2008

Min

ute

s t

o R

esp

ond

Days of Data Collection

Box & Whisker In-House Printing Dept Response Time

• The yellow and aqua areas of each Box and Whisker contain the 2nd and 3rd quartiles of data

• The 1st and 4th quartiles are in the “whiskers”

• The asterisks are outliers (calculated by a formula)

As part of your analysis you try different routes to see if they make a difference graphically

Use the file B&W to generate a Box and Whisker plot◦ Highlight A, B and C

◦ Run Box, Dot & Scatter > Box and Whisker

◦ Select “Columns” on the “Group by…” dialogue box

◦ Click OK on the titles

41

B&W

27.275

32.275

37.275

42.275

47.275

52.275

57.275

62.275

Route 1 Route 2 Route 3

Valu

es

Subgroups

Route 1 - Route 3

What observations can you make?Which appears to be the best route? Why?

42

Measures the strength of association between the input and the output

◦ Y = f(x)

The simplest tool for determining the effect on the output based on a change in the input

Based on your high school math◦ Y = mx + b (where “m” is the slope and “b” is

the y-intercept)

44

Follow along using the file SLR to analyze the drive time versus the number of stop lights hit while taking two different routes to work

Open file SLR

Highlight columns A and B (route 1)

Run Box, Dot & Scatter Plot > Scatter

Click OK through the title slides

Repeat on columns D and E (route 2)

45

SLR

y = 1.575x + 45.033

R² = 0.9923

47

49

51

53

55

57

59

1 2 3 4 5 6 7 8 9

Dri

ve T

ime

Stop Lights Hit - Route 1

Drive Time vs Stop Lights Hit - Route 1

Prediction Equation

R2 - The amount of influence the data points have on the shape of the line

46

y = 2.2703x + 41.568

R² = 0.7063

45

47

49

51

53

55

57

59

61

63

65

2 3 4 5 6 7 8 9

Tim

e

Stop Lights Hit - Route 2

Time vs Stop Lights Hit - Route 2

Compare the graphs. What observations can you make?

47

If I hit no stop lights, which has the shortest drive time?

If I assume that I will hit an average of 5 stop lights each day, how long will the trip take for each route?

If I hit 8 stop lights, which route has the shortest drive time?

48

Use to statistically determine if there are differences between a sample and a target or between two or more sample groups

Used to determine whether making a change to the input variable will result in changes to an output

Without hypothesis testing teams may make adjustments that are not actually required◦ These knee jerk responses can amplify variation

and cause additional problems

50

In manufacturing, you might want to compare two or more raw materials and determine if they produce the same quality

Hypothesis testing helps identify ways to reduce cost and improve quality

51

Define a null (Ho) and an alternative (Ha) hypothesis◦ Ho = the sample is the same as the target or the samples

are the same

◦ Ha = at least one of the samples are different from the target or other sample(s)

There are hypothesis tests for means, medians, proportions, variance and dependence

The goal is to prove that they are not statistically the same at some level of confidence (usually 95%, 99%)

52

Three types of Hypothesis testing1. Classical Method – comparing a test statistic to

a critical value (very statistically oriented)

2. p value Method - the probability of a test statistic being contrary to the null hypothesis

• If the p value is equal to or greater than the a

value (or level of significance), the null hypothesis is confirmed (remember sample sizing)

3. Confidence Interval Method – is the test statistic between or outside of the confidence interval (used for target values)

53

The p-value is the probability that your conclusion of the null hypothesis is incorrect (e.g. your results are highly unlikely to occur in a real world)◦ Keep in mind: the data is not good or bad, it just

does not fit your hypothesis

◦ You may not have enough data in your sample to prove your original hypothesis

54

Run Normality first on ALL variable data◦ T tests and ANOVA are used for normal data

Note: samples are run separately and each sample set must be normal to run these tests

◦ Non-Parametric tests are used for non-normal data

F tests – run equal variance tests when comparing two or more samples◦ Bartlett's – test for equal variance for normal data

◦ Levene's - test for equal variance for non-normal data

55

Normality (Variable)◦ Testing for normality of the data

F Test (variable)◦ Comparing variances (normal and non-normal)

t-Tests (variable)◦ Used for comparing means

ANOVA◦ Used for comparing more than two means

Non-Parametric (variable)◦ Used to analyze non-normal data

Proportion (attribute)◦ Used to analyze proportions

Chi Squared (attribute)◦ A test of the dependency between input and output◦ Excellent for transactional processes

56

Normality – normality of data◦ Statistical Tools > Descriptive Statistics - Normality Test

Tests for Equal Variance of Multiple Samples◦ Statistical Tools > F Test: Two sample for variance (normal

data)◦ Statistical tools > Levene’s test for variance (non normal

data)

Mean◦ 1 sample t test - Statistical Tools: > t test one sample

Comparing one sample to a target

◦ 2 sample t test - Statistical Tools > T test; two sample assuming equal variances Comparing two samples

◦ ANOVA Single Factor – Statistical Tools > Anova single factor Comparing three or more samples

57

Analyze 4 - Hypothesis Roadmap

Non Parametric (for non normal data)◦ Statistical Tools > Stat Templates > 1 sample sign

Comparing sample data versus target

◦ Statistical Tools > Stat Templates > Mann Whitney

Comparing two samples of data

◦ Statistical Tools > Stat Templates > Kruskal Wallis

Comparing three samples of data

58

Proportion Tests◦ Attribute data

◦ Statistical Tools > 1-2 Proportions Test

Chi Square Test◦ Dependence / Independence of the interaction

between inputs and outputs

◦ Statistical Tools > Chi Squared

59

You want to check and see if two different routes have different drive times.◦ Use columns B and C from the file B&W to

determine if they are statistically different

Go to the Statistical Tools > Descriptive Statistics -Normality Test for means

60

B&W

Route 2 Anderson-Darling Data is Normal

38.0 A-Squared 0.359

41.5 p 0.375

42.0 95% Critical Value 0.787


Route 3 Anderson-Darling Data is Normal

31.9 A-Squared 0.561

41.1 p 0.109



Route 2 Route 3

The p-value is preset to be 5% or 0.05 for the normality test; both p-values are greater than 0.05 so we accept our null hypothesis that the

data is normal

61

◦ Run the proper test for equal variances

Go to Statistical Tools > F Test; Two-sample for variance

If one or both were not normal, you would have run Levene’s test for variance instead

Keep the significance at 0.05 and click OK on the titles

62

F-Test Two-Sample for Variances a 0.05

Route 2 Route 3

Mean 37.38 37.12

Variance 16.64178 72.21067

Observations 10 10

df 9 9

F 0.23

P(F<=f) one-tail 0.020 0.040 Two-tail

F Critical one-tail 3.18 4.03 Two-tail

One-tail Reject Null Hypothesis because p < 0.05 (Variances are Different)

Two-tail Reject Null Hypothesis because p < 0.05 (Variances are Different)

Conclusion: The variances are different

Go to Statistical Tools > t Test: Two-sample assuming unequal variances

63

t-Test: Two-Sample Assuming Unequal Variances a 0.05

Equal Sample Sizes

Route 2 Route 3

Mean 37.38 37.12

Variance 16.64178 72.21067

Observations 10 10

Hypothesized Mean Difference 0

df 13

t Stat 0.087

P(T<=t) one-tail 0.466Cannot Reject Null Hypothesis because p > 0.05 (Means are the same)

T Critical one-tail 1.771

P(T<=t) two-tail 0.932Cannot Reject Null Hypothesis because p > 0.05 (Means are the same)

T Critical Two-tail 2.160

Chi Square testing is used to see if the results are independent or dependent on an input◦ The null hypothesis is that they are independent (p-

value > a)

◦ Chi Square testing is excellent for analyzing survey data

Open file Chi Square

Highlight columns A, B, C and D

Statistical Tools > Chi Squared

64

Chi Square

65

21-40 41-60 60+ Total Chi-Sq 18.31879241

Like 27 12 24 63 p 0.001069037

Don’t Care 35 67 31 133 a 0.05

Hate 13 21 17 51 Variables are Related

Total 75 100 72 247

21-40 41-60 60+ Contribution

Like 3.238126084 7.15178716 1.729451835

Don’t Care 0.717948718 3.213296703 1.556929182

Hate 0.399032574 0.006008573 0.306211576

The highest contribution comes from the interaction that is least expected• This gives you clues about the

population

A p-value less than 0.05 tells you that the

variable and the output ARE dependent!

Your historic defect rate has been 5.4%. You have made some improvements and want to see if that is been reduced. You collect 154 samples and there are 5 defects. You declare success based on your new 3.2% defect rate. Based on this sample, have you really made a difference? ◦ Open a blank Excel Spreadsheet◦ Statistical Tools > 1-2 Proportions Test◦ Choose the tab labeled “One Proportion”

Enter 0.946 (success rate) in the yellow box of column A

Enter 154 in the yellow box of column B (trials)

Enter 149 in the yellow box of column C (successes)

Keep the confidence level at 0.95 (a = 0.05)

66

67

Look for the Red colored boxes in the p (normal) area. The sample is NOT different from the historical average!

It shows the p-values for all three of the Possible scenarios:• Sample (H1) = Historic Proportion (H0)• Sample > Historic Proportion• Sample < Historic Proportion

Note the p values are all greater signifying that the Null Hypothesis is true. If any of the p values were less than 0.05 it would signify a relationship that was not true (e.g. H1<H0)

Try gathering More Data… More data gives a more accurate number

0.95 Confidence Level

Proportion Trials Successes Sample p 95% Confidence Intervals p (Direct) p (Normal)

0.946 154 149 0.967532 0.925860 0.989375 0.154 H1<>H0 0.237 H1<>H0

0.93954 0.995525 0.077 H1>H0 0.119 H1>H0

0.923 H1<H0 0.881 H1<H0

Champion Analyze Phase Checklist

68

Analyze 2 – Tollgate and Approval

Analyze Phase CommentsWhat are your deliverables for this phase? Summarize the findings.

Has your Problem Statement or Objective Statement changed? If yes, why?

Have you completed you Fishbone (Ishikawa) analysis to identify variable in our

process?

How many significant (vital few) variables influence the process and what are they?

What sources of variation have been identified?

Have you started your FMEA?

Have you performed any Root Cause Analysis on your process?

Have you done any Graphical Analysis to identify key input varibles in your

process?

Have you completed any regression analysis?

Have you confirmed any findings using Hypothesis Testing? What are the

conclusions?

What is the potential contribution of each of the vital few variables?

What interim actions have you taken to contain defects until a final solution can be

developed and implemented? Has the FMEA been completed?

What tools have you used in this phase and how were they helpful?

What are your improvement plans (containment actions and long term solutions) and

next steps to get there (including timing, responsibility and expected results)?

Has your COPQ changed?

What are you conclusions from this phase?

Are you on track to meet the scheduled completion date?

Are you satisfied with the level of cooperation and support you are getting?

Have you obtained the signatures from leadership to move on to the next phase?

Project Team Analyze Phase Checklist

69

Analyze Tollgate Approval

Champion Approval Signature/Date:

Tollgate review approved unconditionally:

Tollgate review approved with the following contingencies:

Tollgate review dis-approved, list issues for resolution:

identify and analyze possible causes (x’s) identify and

Documents