Review of Reliable DesignQUEST

Frank Federico

What does this line graph tell you?

Education

Standardization

Improvement

• Improvement requires change• Doing the same thing and expecting

different results is insanity (Einstein)

• All improvement requires change; not all change is improvement.

Framework for Reliable Design• Reliability occurs by design not by accident

• Process is the action point of all improvement methodologies

• Segmentation allows the perfection of the design

Starting Labels of Reliability• Chaotic process: Failure in greater than 20% of opportunities

• 80 to 90 % (10-1): 1 or 2 failures out of 10 opportunities

• 95% or better (10-2): 5 failures or less out of 100 opportunities

(These are IHI definitions and are not meant to be the true mathematical equivalent)

Non-Catastrophic Processes

• Definition: Failure of the process does not lead to death or severe injury within hours of the failure

• Less than 95% performance or worse is most commonly seen in these processes (hand-washing as an example)

• Poor outcomes do not occur with each defect due to either to biologic or system resilience

Reasons for the Reliability Gap in Healthcare

• Current improvement methods in healthcare are highly dependent on vigilance and hard work

• The focus on benchmarked outcomes tends to exaggerate the reliability within healthcare hence giving both clinicians and leadership a false sense of security

• Permissive clinical autonomy creates and allows wide performance margins

• The use of deliberate designs to achieve articulated reliability goals seldom occurs

Improvement Concepts Associated with < 95% Performance

(Primarily can be described as intent, vigilance, and hard work)

• Common equipment, standard order sheets, multiple choice protocols, and written policies/procedures

• Personal check lists

• Feedback of information on compliance

• Suggestions of working harder next time

• Awareness and training

Focus on Benchmarks:Biology Protects Us

• All defects in process do not lead to bad outcomes

• Healthcare tends to look at outcomes and not the reliability of the process leading to outcomes (hand washing is an example)

• Benchmark to best practice not aggregate averages

Focus on Clinical Autonomy in Health Care Processes

Desired - variationbased on clinical criteria, no individual autonomy to change the process,process owned from start to finish,can learn from defects before harm occurs, constantly improved by collective wisdom - variation

Current - Variable, lots of autonomynot owned,poor if any feedback for improvement, constantly altered by individual changes, performance stable at low levels

Terry Borman, MD, Mayo Health System

Use of Deliberate DesignThe Reliability Design Strategy

• Prevent initial failure using intent and standardization

• Identify defects (using redundancy) and mitigate

• Measure and then communicate learning from defects back into the design process

The Three Step Design for ReliabilityDesign Techniques Steps

1-Identify the process to standardize2-Segment the population to test the design for anomalies3-Use both hard work and human factor concepts

Prevent initial failure by standardizing the

process to achieve 80% (step 1)

1-Utilise a robust human factors concept to make visible failures from step 1 after step 1 has achieved 80% reliability 2-Once the failure is identified, apply an action to mitigate the failure

Identify failures in step 1 and apply an action to achieve 80% for these failures (step 2)

1-Identify common failures2-Develop a method to measure and study failures3-Utilise knowledge of common failures to redesign either step 1 or step 2

In either step 1 and/or step 2 detect the failures, and use the knowledge from analysis of the failures to redesign (step 3)

Using Segments• Allows for the control of some variables• Defines the boundaries around which sequential

expectations for success can be found• More likely to test the validity of the design rather than

deal with barriers• Fosters a deeper understanding of the design complexity

required for the project• Forces understanding of the differences between

segments as design strategies• Allows the formation of more predictable timelines

Examine Your Own Process

• What is the level of reliability of the process?• Does the success of the process depend on a

person?• Is our main effort to improve the process

training and education?• What will you do to improve the level of

reliability?• Did you begin with a segment and the spread?

The “Set Up” for Reliability• Select a topic whose outcome you want to improve• Determine a high volume segment for initial design

testing• Build a high level flow chart for that segment• Determine where the defects occur in the current

system• Determine where your design work will begin with

by identifying where the commonest defects occur• Your goal is always 95%

Topic: Transition Home

Patient admitted to the unit

Admission assessment completed

Social worker notified to conduct discharge planning assessment

SW assessment completed on day 2 or next business day

Segment: Patient Population/ nursing unit

Social worker not notified

Our aim is to achieve a reliability of 95% with ensuring that the social worker is notified

The Reliability Design Strategy

• Prevent initial failure using intent and standardization

• Identify defects (using redundancy) and mitigate

• Measure and then communicate learning from defects back into the design process

Why Standardize?

• Contributes to building an infrastructure (who does what, when, where, how and with what)

• Support training and competency testing to sustain the process

• Achieve front line articulation of key processes by staff

• Allows the appropriate application of Evidence Based Medicine consistently

• Feedback about defects and application of learning to design is possible

Your Task

• What will you standardize?• How will you engage front line staff?• How will you know if the change is an

improvement?

Next stepDetection-Mitigation

• Develop a redundancy/contingency/back-up

• Must be independent to be effective• Must be careful as it is a use of resources

Human Factor Concepts

• Decision aids and reminders built into the system

• Desired action the default (based on evidence)

• Redundant processes

• Use fixed current scheduling in design

• Take advantage of habits and patterns• Standardization of process based on clear specification

and articulation

Human Factors and Reliability Science:(Designing sophisticated failure prevention, failure identification and mitigation)

Characteristics of “Redundancy Tools”

• Require careful consideration since they do represent a form of “waste”

• Needs to be connected to the process almost all the time (at least 10-1)

• Requires a good standardization functions be in place before implementing a redundancy

• Need to be truly independent• Need to be used or will no longer function as

a good filter• Must follow with a mitigation strategy

Measurement

• Measure and then communicate learning from defects back into the design process

• Measurement should take place at each step

• Measurement should be as close to real time as possible

• If the system is not performing as designed must make changes

As you begin your work

• Have we selected a segment?─What advantage does this segment provide?

• Develop a high-level flow diagram• Identify the most common defect• Standardize: use small tests of change• Measure the reliability of your process

Your Next Task

• What redundancy/back-up will you have in place?

• Remember that this process must also be standardized.

When to Re-design

• If you have a process and it is not reliable ask:─Is it truly standardized?─Why are staff not using the process?─What is failing?─Is it time to develop a totally new process?

Key Questions To Analyze Testing and Implementation

Key Question Your EvaluationIs the connection between goals and process clear?

Is the design strategy primarily vigilance and hard work?

Has some degree of segmentation been used to test the design?

Is standard work with testing been part of the design?

Is a design methodology being used?

Are small tests of change being used in a rapid cycle?

Is data collection rapid enough?

Key Questions To Analyze Spread

Key Question Your EvaluationHave you repeated the small test cycles as you spread from the initial site

Is the process of spread dependent on one person

Has some degree of segmentation been used to spread

Has customization been allowed or encouraged

Is the same team who developed the pilot now responsible for spread

Have you shifted your focus from process reliability to outcomes too early