reducing reagent waste through process improvement and
TRANSCRIPT
Reducing Reagent Waste Through Process
Improvement and Preventive Maintenance
By
Amy Rose Gobel
A.B., Princeton University 2012
Submitted to the MIT Sloan School of Management and the Institute for Data, Systems, and
Society in partial fulfillment of these requirements for the degrees of
Master of Business Administration
and
Master of Science in Engineering Systems
in conjunction with the Leaders for Global Operations Program at the
MASSACHUSETTS INSTITUTE OF TECHNOLOGY
June 2017
© Amy Rose Gobel, MMXVII. All rights reserved.
The author hereby grants to MIT permission to reproduce and to distribute publicly paper and electronic copies of this thesis document in whole or in part in any medium now known
or hereafter created.
Author .................................................................................................................................
MIT Sloan School of Management and the Institute for Data, Systems, and Society May 12, 2017
Certified by .......................................................................................................................... Nikos Trichakis, Thesis Supervisor
Assistant Professor, MIT Sloan School of Management
Certified by .......................................................................................................................... Dan Whitney, Thesis Supervisor
Senior Research Scientist Emeritus, MIT Leaders for Global Operations
Approved by ........................................................................................................................ Maura Herson
Director, MBA Program, MIT Sloan School of Management
Approved by ........................................................................................................................
John N. Tsitsiklis Clarence J. Lebel Professor of Electrical Engineering, IDSS Graduate Officer
THIS PAGE IS INTENTIONALLY LEFT BLANK.
3
Reducing Reagent Waste Through Process Improvement and Preventive Maintenance
by
Amy Gobel Submitted to the MIT Sloan School of Management and the Institute for Data, Systems, and Society and the on May 12, 2017, in partial fulfillment of the requirements for the degrees of
Master of Science in Engineering Systems and
Master of Business Administration Abstract Quest Diagnostics has committed to reducing operating expenses by $1.3B between 2012 and 2017. A portion of the cost-saving initiative focuses on reagents – expensive liquids that are combined with patient samples to detect compounds of interest. This project aims to reduce reagent waste for high-volume diagnostic tests run on an instrument platform that generates a relatively high amount of reagent waste. Waste, in this context, means any reagent that does not generate unique patient results. Therefore critical components of the quality system, such as quality control and calibration tests, are designated waste even though they are a necessary expenditure. Quality control (QC) samples and mechanical errors accounted for 5.2% and 4.4%, respectively, of all reagent usage prior to the start of the project. Mechanical errors occur when the diagnostic testing platform encounters something unexpected, such as debris or a reading that indicates insufficient sample volume, which interrupts sample processing. The instrument jettisons this test and attempts to repeat the assay. Initial discussions with laboratory representatives revealed differing interpretations of quality control requirements. All sites using the platform of interest were then surveyed to gauge the extent of variation. All sites met quality control requirements but several exceeded them. The most pertinent variations are listed below.
1. Frequency: Several sites ran control samples more often than established in Standard Operating Procedure (SOP) requirements, increasing total QC usage by over 70%.
2. Container size: The choice of container determines the amount of “dead volume”, material that the instrument cannot access and must be discarded. Some sites used containers with 12.8 times the dead volume required in the smallest option.
3. Reuse policy: Some labs reuse containers of quality control materials across multiple batches. Reusing QC material further reduces the amount of dead volume discarded, but using new QC materials eliminates the possibility of evaporation between batches.
An interdisciplinary team of experts tasked with maintaining the SOPs has reviewed these results and will clarify the appropriate SOP interpretation to unify practices across laboratories.
4
In order to understand mechanical errors, I observed routine maintenance at four sites and found that business units did not consistently share best practices. Collaborating with vendor representatives and operators, I launched an Autonomous Maintenance (AM) pilot program in order to develop training materials capturing institutional knowledge and to test additional maintenance procedures. The AM activities generated 29 training documents, which were added to a national database of competency training materials. All operators certified to operate the testing platform will be required to review and pass comprehension quizzes on the training materials. As the Marlborough site continues to develop improvements to the maintenance procedures, these changes will be shared with the vendor and incorporated into training documents. Thesis Supervisor: Nikos Trichakis Title: Assistant Professor, MIT Sloan School of Management Thesis Supervisor: Dan Whitney Title: Senior Research Scientist Emeritus, MIT Leaders for Global Operations
5
Acknowledgements
It takes a village to raise a thesis. First and foremost, I would like to
thank Toni Kick for mentoring me through all dimensions of my time with Quest
Diagnostics. Thanks go as well to Carlene Wong and Todd Raymond for making
sure I was at home in the Marlborough office, to Craig Vorwald for his guidance
on the nuances of quality control procedures, and to Brian Dunn for the constant
support in learning TPM.
This thesis was only possible because of the support, access, and guidance
I received from the entire team at Marlborough, especially Denis Gallagher, Mike
Hellyar, Margherita Walkowski, Paula Arnone, Cynthia Lam, Phil Chalvire, Dan
Carty, Sean Spooner, and ace operator MaryEllen Marshalsea. What I learned at
Marlborough was magnified by the opportunity to visit the labs in Miramar,
West Hills, and Wood Dale, and I am in debt to those teams for their patience as
I learned from their innovations. I also owe countless thanks to the vendor team
for their support in the AM initiative and for answering my countless questions
about the platform.
On the home front, I thank the LGO team for letting me be one of the
lucky 45 to experience this challenging, enlightening program, and for supporting
me throughout these 24 months. I would also like to thank Jason Jay and the
MIT Sloan Sustainability Initiative for helping me apply a sustainability lens to
the process improvement activities.
Then of course I owe so much to my advisors: to Dan Whitney for asking
the right guiding questions and sharing his experience in the diagnostics industry,
and to Nikos Trichakis for taking a chance on advising an LGO thesis.
I send warm thanks to the LGO and Sloan friends I have gotten to know
and love over the last glorious two years. This community is unparalleled in
richness, and I am honored and humbled every day to be a part of it. Finally
with deep appreciation I thank my parents for their unquestioning, unwavering
support through all my adventures.
6
THIS PAGE IS INTENTIONALLY LEFT BLANK.
7
Contents 1 Introduction 11
1.1 Problem statement .............................................................................. 11
1.2 Overview of approach, major findings, and recommendations ............ 12
2 Company context 16
2.1 Quest Diagnostics business model ....................................................... 16
2.1.1 Value proposition ................................................................... 16
2.1.2 Regulatory context ................................................................ 18
2.2 Competitive landscape ......................................................................... 19
2.3 Recent performance and history of Invigorate savings ........................ 20
2.4 Summary of relevant organization structure ....................................... 20
2.5 Chapter summary ................................................................................ 21
3 Total Productive Maintenance overview 22
3.1 Brief history of TPM ........................................................................... 22
3.2 Summary of goals, pillars, wastes, and primary metrics ..................... 23
3.3 Drivers of success and failure .............................................................. 27
3.4 Chapter summary ................................................................................ 28
4 Process description 29
4.1 Description of instrument .................................................................... 29
4.1.1 Analytical principle ................................................................ 29
4.1.2 Main mechanical systems ....................................................... 30
4.1.3 Mechanical functions and common errors .............................. 32
4.1.4 Reagent waste ........................................................................ 35
4.2 Maintenance overview ......................................................................... 36
4.2.1 Operator maintenance ........................................................... 36
4.2.2 Vendor maintenance .............................................................. 37
4.3 Stakeholder analysis ............................................................................ 37
4.3.1 Primary groups and their interests ........................................ 37
4.3.2 Potential conflicts .................................................................. 39
4.4 Chapter summary ................................................................................ 39
5 Current state analysis 41
5.1 Process observation ............................................................................. 42
5.1.1 Activities completed .............................................................. 42
5.1.1.1 Observed components of routine operation .............. 42
5.1.1.2 Prepared quality control survey ............................... 44
5.1.2 Preliminary observations ....................................................... 45
5.2 Data collection and evaluation ............................................................ 48
8
5.2.1 Sources of data ....................................................................... 48
5.2.2 Preliminary observations ....................................................... 50
5.3 Chapter summary ................................................................................ 53
6 Countermeasures to eliminate quality control waste 54
6.1 CLIA regulations regarding quality control procedures ...................... 55
6.2 Quest Diagnostics quality control requirements .................................. 55
6.3 Results from quality control survey .................................................... 57
6.4 Tension between quality requirements and reagent cost ..................... 61
6.5 Implementing quality control procedure changes ................................ 67
6.5.1 Changes at an example laboratory ........................................ 67
6.5.2 Savings due to changes .......................................................... 67
6.5.3 Continuing to align quality control practices ........................ 68
6.6 Chapter summary ................................................................................ 69
7 Countermeasures to eliminate mechanical error waste 71
7.1 Goals for AM pilot program ................................................................ 71
7.2 CLIA regulations related to operation, maintenance, and equipment
modifications ............................................................................................. 72
7.3 AM team structure and activities ....................................................... 73
7.3.1 Team structure ...................................................................... 73
7.3.2 Autonomous Maintenance activities ...................................... 74
7.4 Outcomes ............................................................................................. 80
7.4.1 Impact on instrument performance at Marlborough .............. 80
7.4.2 Description of training materials and how training materials
were disseminated ........................................................................... 81
7.5 Discussion of implementation approach .............................................. 81
7.5.1 Challenge of improving processes in a lab with high
productivity goals ........................................................................... 81
7.5.2 Strategy for sustaining improvements ................................... 83
7.6 Chapter summary ................................................................................ 83
8 Conclusions 85
8.1 Summary of main findings ................................................................... 85
8.2 Recommendations for Quest Diagnostics ............................................ 86
8.2.1 Specific process/decision-based recommendations ................. 87
8.2.2 Organizational recommendations ........................................... 87
8.2.3 Data collection recommendations .......................................... 89
8.3 Areas of future investigation ............................................................... 90
9
List of Figures
2.1 Process map of Quest Diagnostics service ...................................................... 17
4.1 Map of main mechanical systems in instrument ............................................ 30
4.2 Path of sample processing within instrument ................................................ 32
6.1 Rate of quality control use in example laboratory ......................................... 68
7.1 Example One-Point Lesson ............................................................................ 75
7.2 Rate of mechanical errors in Marlborough ..................................................... 80
List of Tables
5.1 Reagent usage March-May ............................................................................. 51
5.2 Error code data for an example laboratory March-May ................................ 52
6.1 Frequency of quality control use .................................................................... 57
6.2 Quality control material acceptance criteria .................................................. 58
6.3 Reuse policy and container size for quality control material ......................... 58
6.4 Reused quality control material storage policies ............................................ 60
6.5 Approach to mechanical errors on quality control samples ............................ 61
10
THIS PAGE IS INTENTIONALLY LEFT BLANK.
11
Chapter 1 Introduction
This thesis aims to identify and eliminate the sources of unnecessary reagent
consumption on a high-volume diagnostic testing platform at Quest Diagnostics. The
leading sources of avoidable reagent use are excessive quality control and mechanical
errors. Quality control tests are a necessary part of testing procedures, and
regulations require a minimum frequency and type of quality control. Mechanical
errors arise when an abnormality such as debris or an unexpected reading interrupts
normal sample processing. Observations and analysis of instrument performance data
lead to two main findings:
1. Unnecessary quality control use arises when individual labs implement quality
control practices that go above and beyond company or regulatory
requirements; and
2. Mechanical errors arise because the manufacturer’s recommended
maintenance and training procedures do not sufficiently address debris
accumulation in the instrument.
Given these findings, the proposed countermeasures included establishing clearer
standards for quality control practices and increasing the intensity, frequency, and
consistency of maintenance activities beyond manufacturer requirements. In addition
to exploring the impact of these countermeasures, this work highlights the challenges
of maintaining consistency across a large company and promoting a culture of
continuous improvement in an environment with high productivity goals.
1.1 Problem statement Quest Diagnostics generates diagnostic information that allows individuals and
physicians to make informed decisions about healthcare options. The company
performs diagnostic testing in 29 domestic and four international laboratories, each
drawing patient samples from a large region – for example, an area including most of
New England – through a reverse logistics network. Annually the company serves
approximately one third of the US adult population, generating $7.5B in revenue.
12
Quest Diagnostics offers a wide range of tests, from the standard array of blood tests
performed for routine physical examinations to esoteric, highly specialized genetic
analyses. Because of its breadth of offerings and relatively low cost, the company
holds substantial market share within the field of diagnostics. However, it operates in
a shifting, highly competitive landscape. Competition from private physician
practices, in-hospital laboratories, and large commercial laboratories threaten to take
over business, and decreasing reimbursement rates from governmental healthcare
payers put downward pressure on revenues.
Within this challenging context, Quest Diagnostics launched an initiative called
Invigorate in 2012 to generate $1.3B in run-rate operating cost savings. One
component of the initiative focuses on reagents, expensive and highly specialized
liquids that are combined with patient samples in order to detect compounds of
interest. This thesis aims to reduce unnecessary reagent consumption on a high-
volume diagnostic testing platform.
1.2 Overview of approach, major findings, and
recommendations This section provides a brief overview of the following chapters, highlighting the
methodology, findings, and recommendations of the project.
Quest Diagnostics operates in a complex competitive landscape within a highly
regulated industry. Chapter 2 provides the relevant financial, industrial, and
regulatory context. Financially, the company has struggled to consistently increase
revenues at the desired rate over the last five years, leading to ever-increasing
scrutiny on operating costs. The Invigorate initiative generates savings from many
sources, including reducing reagent waste – the focus of this thesis – and increasing
employee productivity.
The company’s competition includes similar large-scale commercial laboratories,
laboratories located within hospitals, and laboratories run in private physician
practices. The two latter types of competitors generally offer faster turnaround times
because of the relatively short transportation between sample collection and sample
analysis, as both can be performed within the same building. However, they
generally operate at higher cost and offer a narrower range of test options. As a
result, Quest Diagnostics competes based on the price and the breadth of its
offerings while maintaining a competitive turnaround time for sample results.
13
The changes that the company can make in the interest of increased efficiency are
bounded by the highly regulated nature of the diagnostics industry. The Centers for
Medicare & Medicaid Services (CMS) oversees the implementation of the Clinical
Laboratory Improvement Act (CLIA), which sets standards for minimum
maintenance and quality control procedures. CMS has granted enforcement
authority to several non-governmental organizations, such as the College of
American Pathologists (CAP), that inspect and accredit laboratories such as those
operated by Quest Diagnostics.
Each Quest Diagnostics laboratory operates many testing platforms. Each
instrument represents a highly complex piece of mechanical and biomedical
engineering. As such, adequate maintenance plays a key role in ensuring quality test
results and meeting turnaround time expectations. Chapter 3 discusses the history
and main principles of Total Productive Maintenance (TPM), a philosophy of
maintenance developed in manufacturing settings in Japan starting in the early
1970s. TPM emphasizes proactive rather than reactive maintenance and empowers
front-line workers to develop improved maintenance procedures.
Chapter 4 then describes the processes related to the testing platform of focus.
Instrument operators are responsible for running patient and quality control samples;
calibrating the instrument to ensure accuracy over time; performing routine
maintenance; and troubleshooting mechanical errors that arise during any of the
other steps. To understand the instrument function, it is important to first
understand the ten major mechanical subsystems within the instrument, their failure
modes, and the maintenance for each.
The chapter also describes the main stakeholders related to the instrument and by
association to the project. Front-line operators play a key role because they perform
the main functions related to the instrument. Laboratory management makes key
decisions related to how operators spend their time, given performance goals
established by regional management. The operations occur within the context of
manufacturer and company-specific standards. Specifically, the instrument
manufacturer establishes a set of minimum operating requirements. Then a national
Best Practices Team (BPT) within Quest Diagnostics uses this baseline to establish
Standard Operating Procedures (SOPs) for the company.
While the main groups related to this project are united around the goal of providing
accurate patient results, their unique perspectives and constraints lead the main
organizational challenges of this project, specifically the tensions between
14
productivity goals and laboratory improvement initiatives, benefits of more
conservative quality control practices and costs of excessive reagent use on quality
control testing and manufacturer intentions and their capabilities.
Having established the background of the company, methodology, and process,
Chapter 5 discusses the current state of the diagnostic testing platform of interest.
The current state analysis involved direct observation of four laboratories using the
platform and data related to instrument usage and performance. The main findings
were as follows:
(1) Excessive quality control and mechanical errors – that is, interruptions to
sampling processing – were the largest sources of avoidable reagent usage;
(2) Individual laboratories implemented quality control practices that exceeded
Quest Diagnostics operating procedures and regulatory requirements;
(3) Mechanical errors on the instruments were a major source of irritation for
operators;
(4) The most common mechanical errors may be addressed through increasing
maintenance beyond manufacturer requirements; and
(5) Laboratories do not consistently share best practices related to maintenance or
process improvements.
Chapter 6 reviews this project’s approach to the first two observations above. A
survey of all Quest Diagnostics laboratories using the target platform established
that quality control practices differ in frequency, acceptance criteria, reuse policy,
storage approach, container size, and approach to mechanical errors. Individual
laboratories established their own balance between the benefits and the costs of
quality control measures beyond those required by company SOPs and regulatory
requirements, leading to substantial differences in how much quality control material
and reagents laboratories used in their quality control procedures. The chapter
discusses the pros and cons of the various laboratory policies.
Ultimately, the BPT should establish quality control policies to a level of specificity
that encompasses the varying parameters observed across laboratories. Determining
the reliability of quality control policies, such as reusing material between batches,
may require further investigation. The potential reagent cost savings from adopting
a less conservative policy set a reasonable bound on the budget for such testing.
Just as Chapter 6 addresses the first leading source of waste, quality control,
Chapter 7 examines the second, mechanical errors. Preliminary data suggests that
15
maintenance beyond the manufacturer’s recommendations could reduce the rate of
mechanical errors. As discussed in Chapter 3, the maintenance philosophy of TPM
provides an established structure within which a company may improve maintenance
procedures. Because of the proven success of TPM methods and the accessibility of
an internal expert on the approach, this thesis implements a component of TPM
called Autonomous Maintenance (AM) on the platform within a Quest Diagnostics
laboratory.
An AM team consisting of the author, an instrument operator, and a rotating group
of representatives from the manufacturer, creates two types of outputs to address the
mechanical errors observed. First, we develop a series of training documents called
One-Point Lessons (OPLs) to capture and easily disseminate best practices related to
instrument operation, troubleshooting, and maintenance. Second, we establish a
process for testing supplementary maintenance procedures. In order to fully realize
the benefits from this work, Quest Diagnostics should include the OPLs as part of
operator competency training and continue to create space for ongoing testing of
additional maintenance procedures.
Chapter 8 consolidates the observations and leading recommendations. In addition
to the process-specific recommendations discussed in Chapters 6 and 7, Quest
Diagnostics should consider data collection and organizational changes that will
encourage greater consistency across laboratories and drive a culture of continuous
improvement. To facilitate the ongoing process improvement, Quest Diagnostics
should collaborate with the instrument manufacturer to develop consistent,
streamlined methods of tracking instrument performance and correlating individual
mechanical errors with reagent waste.
At an organizational scale, the company should foster innovation by providing more
incentives for process improvement that increase safety, promote efficiency, or reduce
costs, which operators can submit through the BPT. As laboratories continue to
innovate, the company should continue to perform this type of comparative analysis
to identify other variations that inevitably arise between labs. Finally, the BPT
should update SOPs and training materials through their standard review procedure
to incorporate the process improvements in a standardized way to ensure compliance
across the company. This type of structural change can apply more broadly to any
national organization with processes repeated in multiple locations within the
company.
16
Chapter 2 Company Context This chapter discusses the relevant company context to ground the subsequent
chapters in this thesis. Quest Diagnostics provides clinical diagnostic test results for
approximately one third of the adult population of the United States every year
through an extensive network of regional laboratories, dedicated transportation, and
patient sample collection centers. Despite their large market share, they face
significant competition and an industry context of decreasing reimbursement rates
for testing.
This challenging financial environment led Quest Diagnostics to focus on operational
excellence as a core part of their business strategy starting in 2012. The strategy
includes ambitious cost-saving targets achieved in part by reducing laboratory
headcount and eliminating waste. One component of the cost-savings initiative
targeted reagent waste, thus motivating this thesis. In addition, the company
context of increasing productivity requirements for laboratory personnel created the
defining organizational constraints within which this thesis operated.
2.1 Quest Diagnostics business model
2.1.1 Value proposition
Quest Diagnostics provides diagnostic information services that allow individuals and
physicians to make informed decisions about healthcare. Examples of this type of
information include analyses of cholesterol levels, indications of infectious diseases,
evidence of illegal drug usage, genetic data about predisposition to diseases, and
many others.
The process through which a patient interacts with Quest Diagnostics generally
proceeds is depicted in Figure 2.1 and described here.
1. A doctor needs information about certain aspects of a patient’s health and
orders one or several tests.
17
2. The patient then provides a type of sample as specified by the test
requirements, such as blood, urine, saliva, or exhaled air. The sample is
collected in a dedicated testing area in a doctor’s office or another commercial
location, including spaces that Quest Diagnostics owns and operates.
3. Quest Diagnostics operates a reverse logistics network that collects patient
samples from these locations daily and transports them to the nearest regional
laboratories that can perform the required testing.
4. Once in the lab, the sample is logged and transported to the relevant testing
area.
5. Instrument operators then process the sample as necessary and load the
sample into the appropriate instrument, as specified by the prescribed test.
6. The instrument generates diagnostic information. The operator reviews the
results to confirm that the results meet Quest Diagnostics quality
requirements. If there are no unusual patterns, trends or distributions in
patient results, the operator then releases the results to the prescribing
doctor. Otherwise, the operator repeats testing until acceptable results are
achieved.
7. Meanwhile, the laboratory archives the patient sample for a time period
prescribed by Quest Diagnostics policy in the event that additional testing
becomes necessary.
8. Finally, the doctor uses this information to determine the appropriate course
of action for the patient.
Figure 2.1: Process map of Quest Diagnostics service
At doctor’s office and/or designated sample collection
location
Within Quest Diagnostics regional laboratory
1. Doctor orders test(s) for a patient
2. Patient delivers sample at doctor’s office or other designated location
3. Quest Diagnostics collects sample and transports to regional laboratory
4. Laboratory employee logs sample and transports it to relevant testing area
5. Instrument operator runs the sample on the designated testing platform
6. Instrument operator reviews test results for quality then releases results
7. Laboratory employee archives patient sample
8. Doctor uses laboratory results to determine course of treatment for patient
18
Quest Diagnostics operates primarily in the United States, with 29 domestic
laboratories, including Puerto Rico. In addition, the company operates four
international laboratories in Mexico, India, and Brazil. This thesis will consider only
the domestic laboratories because of the shared regulatory environment and greater
similarities in operations.
This analysis focuses on the stages during which the operator interacts with the
instrument and patient sample to generate diagnostic results – steps 5 and 6 in
Figure 2.1. While other components of the Quest Diagnostic process involve
opportunities to reduce waste, they lie beyond the scope of this analysis.
Each laboratory operates a variety of testing platforms based on the capabilities
required within that region. While the mix of platforms may vary between labs, any
given type of test will always be run on the same testing platform. A single platform
may have the ability to run several different tests.
The instruments themselves represent significant investments in terms of the
immediate cost of the instrument, ongoing service/maintenance costs, and the
process of introducing a new piece of equipment into the laboratory. In 2015,
laboratory equipment accounted for 44% of the company’s property, plant, and
equipment assets [1].
Depending on the platform, Quest may choose to buy the equipment or lease it from
the vendor. In either case, the company may also decide to take on a service contract
with the equipment manufacturer. As a result, the instrument manufacturer usually
maintains close ties with a laboratory that acquires the manufacturer’s instrument
by providing routine maintenance, validated instrument modifications, and/or
ongoing operator training. The relationship with the instrument manufacturer
proved critical to this analysis because of the technical expertise they could provide
and the regulatory requirements that the manufacturer approve any process changes.
2.1.2 Regulatory context
The Clinical Laboratory Improvement Act (CLIA) of 1967 and the subsequent
Clinical Laboratory Improvement Amendments of 1988 established the primary rules
within which Quest Diagnostics operates in the United States. This law established
laboratory-specific guidelines (42 CFR 493), designated the Centers for Medicare &
Medicaid Services (CMS) as the regulatory agency legally and financially responsible
19
for enforcement, and established Medicare and Medicaid compensation guidelines for
tests [2]. CMS has granted deemed status to several non-governmental agencies,
which allows the organizations to perform inspections and accredit laboratories.
Quest Diagnostics laboratories receive accreditation from the College of American
Pathologists (CAP), an association of peer pathologists and laboratory employees.
The FDA regulates manufacturers of In Vitro Diagnostic products (IVDs). The
FDA has asserted that it has jurisdiction over laboratory developed tests (LDTs),
but presently has not regulated LDTs since they are regulated by CMS under
CLIA 1988. There are various legislative proposals that could allow the FDA to
regulate certain aspects of LDT development under three of the FDA Quality
System Regulations (QSRs), whereas IVD and other medical device
manufacturers are subject to approximately 25 QSRs. However, such legislation
and its regulatory implementation are at least six or more years in the
future. Quest Diagnostics does not do animal research testing and does not have
any laboratories that are subject to 21 CFR Part 58 regulations pertaining to
animal and basic pharmacology and toxicology research testing.
Chapters 6 and 7 will discuss the specifics of the regulations as they apply to this
project.
2.2 Competitive landscape Quest Diagnostics operates in a highly competitive landscape in an era of significant
downward pressure on prices. The company faces competition from other commercial
laboratories, such as the company’s largest competitor, Laboratory Corporation of
America; laboratories that operate within hospitals; and private physician
laboratories. The physician-owned and hospital-based laboratories can deliver results
rapidly because of the minimal amount of sample transportation required, so Quest
faces substantial pressure to test patient samples rapidly to compensate for the
additional time spent in transit. Quest provides the additional value of offering a
larger array of testing options than most hospitals or independent labs can
accommodate and providing results at a lower cost because of the scale of their
operations.
Quest also faces decreasing prices. Part of the Affordable Care Act of 2010 involved
decreasing prices that CMS paid for diagnostic services by 1.75%, decreasing the
profit margins that Quest could generate from Medicare and Medicaid patients [1].
20
Private insurance companies have followed suit by decreasing reimbursement rates,
further affecting testing revenues.
2.3 Recent performance and history of
Invigorate savings In this highly competitive context, Quest Diagnostics has struggled to maintain
consistent growth of revenue, income, and margin since 2010. To address the
financial turbulence in May 2012, the company brought on a new CEO, Steve
Rusckowski, with the goal of establishing the company on firmer footing.
Mr. Rusckowski addressed this challenge by establishing a five-point strategy of
restoring growth, reducing operating costs, removing unnecessary layers of
management, divesting of operations outside the core diagnostics information
business, and increasing shareholder dividends. The effort to reduce operating costs,
dubbed the Invigorate initiative, originally aimed to decrease annual operating costs
by $500 million per year by 2014 from a 2011 baseline. After reaching a savings rate
of $700 million in 2014, the goal further increased to a total of $1.3 billion in annual
savings by the end of 2017.
One component of Invigorate focused on reducing waste of reagents – the expensive
liquids combined with patient samples in testing procedures. The reagent savings
target motivated this thesis.
Another important source of savings comes from increasing productivity. Generally
this means increasing laboratory throughput while maintaining or decreasing
employee headcount. While productivity considerations play a role in this thesis,
they are nominally out of scope.
2.4 Summary of relevant organizational
structure To understand the rest of this analysis, we must first review decision-making
processes at a site level and a national level.
Quest laboratories operate in accordance with national Standard Operating
Procedures (SOPs). The groups that establish SOPs, the Best Practices Teams
(BPTs), consist of medical, regulatory, and laboratory experts from across the
company. Each BPT owns the SOPs for a set of clinically related testing procedures.
21
Individual laboratories then have responsibility for implementing the SOPs.
Laboratory Medical Directors may provide guidance about SOP interpretation at a
regional level. As discussed in Chapter 6, by delegating some of the burden of SOP
interpretation, the BPTs allow room for individual laboratories to develop local
practices that exceed Quest Diagnostics standards and regulatory requirements,
leading to unnecessary reagent usage.
Within individual labs, instrument operators run the diagnostic tests, review the
instrument output to confirm that results meet company quality standards, and
release results to the relevant physicians. Within a clinically related area of the lab,
all operators report to a shift supervisor. The supervisors for one or several areas
then report to a laboratory manager. Finally, the laboratory managers report to a
laboratory director. The laboratory director controls decisions related to general
policies or lab targets such as headcount or overtime allocation, but many day-to-
day decisions about the specifics of lab operations often happen between operators
and supervisors. As discussed in Chapter 7, this localized decision-making led
individual labs to create innovative process improvements or develop localized
expertise in some part of instrument operations, but that knowledge was not
consistently shared beyond the lab. Subsequent chapters discuss the efforts of this
thesis to encourage labs to share that knowledge.
2.5 Chapter summary Quest Diagnostics operates in a highly competitive industry in an era of decreasing
prices, and as a result, the company has increasingly focused on decreasing operating
costs as a means of increasing net income. The Invigorate cost-saving initiative
motivates this thesis by setting targets for reducing unnecessary reagent use in
diagnostic testing. Any changes to current operations must be performed within the
bounds of relevant regulations, primarily the Clinical Laboratory Improvement Act,
and the changes must be approved by the company’s internal Best Practices Team
to ensure consistent approaches.
Quest Diagnostics uses highly complex biomedical equipment to generate patient
results, and maintaining these instruments is a critical component of generating
consumer value and abiding by regulatory requirements. The next chapter provides a
brief overview of maintenance philosophies including Total Productive Maintenance,
an approach applied in this thesis to improve instrument performance.
22
Chapter 3 Total Productive Maintenance overview This project aims to reduce waste on a diagnostic testing platform used in many
Quest Diagnostics laboratories. Routine maintenance plays a critical role in
instrument performance, especially in preventing mechanical errors that lead to
reagent waste. This project specifically employed Autonomous Maintenance, a
defining element of the maintenance philosophy Total Productive Maintenance
(TPM). To illuminate why TPM was selected as a countermeasure, this chapter
discusses the history of TPM, its main principles, and the drivers of its success and
failure.
3.1 Brief history of TPM The role of maintenance in manufacturing has evolved significantly over the 20th
century. This section briefly discusses the main predecessors to TPM.
• Breakdown Maintenance: Prior to the 1950s, most manufacturing organizations
performed maintenance only when operating conditions had noticeably
deteriorated [3]. This reactive approach led to expensive, unplanned interruptions
to operations.
• Preventive Maintenance: Introduced in 1951, Preventive Maintenance involves
monitoring the health of the equipment and performing maintenance functions
like lubrication and minor repairs at set times throughout the day, instead of
waiting to perform maintenance once a breakdown has occurred [3].
• Predictive Maintenance: Like Preventive Maintenance, Predictive Maintenance
attempts to address incipient errors before they lead to instrument breakdowns.
Unlike the time-based schedule of Preventive Maintenance, it instead uses
23
physical indicators such as instrument temperature, vibration, or component
wear to determine when to perform maintenance.
• Maintenance Prevention: This approach, introduced in the 1960s, involves
designing or modifying instruments to reduce maintenance steps or the time
required to perform those steps [3].
• Productive Maintenance: This philosophy combines Predictive Maintenance and
Maintenance Prevention with a particular emphasis on maximizing the
productivity of equipment and the lifetime cost of maintenance.
• Total Productive Maintenance: In 1971, the Toyota Motor Company supplier
Nippondenso Company developed Total Productive Maintenance [4]. This
philosophy expands on Productive Maintenance by including the entire workforce
in support of improved instrument productivity. Since that point, TPM has been
treated as a necessary precondition for maintenance excellence. For example, the
Japan Institute for Plant Maintenance began recognizing plants with excellent
maintenance practices in 1964, but since 1971 only plants that had fully
implemented TPM have received the prize [4].
The components of TPM evolved since its foundation. In the late 1970s, the Chuo
Spring Company began delegating the responsibility of process improvement to small
groups within. Building off this success, the Tokai Rubber Company developed the
seven-step Autonomous Maintenance process that now constitutes a foundational
component of TPM [5].
3.2 Summary of goals, pillars, wastes, and
primary metrics The key elements that define a TPM program are as follows:
(1) Focus on maximizing Overall Equipment Effectiveness;
(2) Develop a company-wide preventive maintenance process;
(3) Engage designers, operators, and maintenance personnel in improving
equipment;
(4) Involve all levels of company up to top managements; and
(5) Implement preventive maintenance through autonomous small-group activities
[6].
These components combine to accomplish the following goals:
24
(1) Achieve zero defects;
(2) Run instruments at 100% of available capacity;
(3) Prevent the “six big losses” of instrument inefficiency;
(4) Give operators ownership of equipment maintenance;
(5) Improve operators’ abilities;
(6) Design instruments that minimize maintenance, breakdowns, and the time
between installing and achieving full functionality.
Zero defects: Realistically, no plant can achieve zero defects. However, by striving for
100% quality, TPM encourages operators to identify minor problems before they
escalate into larger, more impactful problems.
100% available capacity: The primary metric for instrument performance is Overall
Equipment Effectiveness (OEE), which is defined as follows:
OEE = Availability x Performance x Quality.
In turn,
Availability = (time instrument was running) / (time available for instrument
to run),
where the difference between the numerator and the denominator arises from
unplanned breakdowns that reduce the total amount of time the instrument can
function during a day. This number could also be reduced when critical parts are not
delivered on time, forcing the instrument to remain idle.
Then, given the total amount of time available to run,
Performance = (actual # jobs) / (ideal # jobs).
The ideal number of jobs is the theoretical instrument throughput in the allocated
time, given the theoretical minimum cycle time. Finally, given the number of jobs
performed,
Quality = (# successful jobs) / (# jobs).
A world-class manufacturing facility will achieve Availability ¿ 90%, Performance ¿
95%, and Quality ¿ 99%, for an OEE ¿ 85%. Plants that receive recognition from
JIPM generally have an OEE exceeding 90%, demonstrating the difficulty of
achieving high values for this metric [4].
25
Chapter 5 discusses the difficulty of establishing a consistent method of determining
OEE for the target testing platform. Ultimately, the project relied on weekly quality
data and information about the frequency of mechanical errors to track instrument
performance over time.
Six big losses: TPM identifies six categories of problem that affect OEE. These “six
big losses” are as follows:
1. Breakdown: When a mechanical failure has forced an instrument to stop
processing, TPM identifies the lost time as a breakdown loss.
2. Setup and adjustment: This type of loss includes the time spent changing
instrument parameters, swapping out dies, or otherwise preparing the
instrument for function when it is not in a breakdown state.
3. Idling and minor stoppage: Small errors, such as a material jam or blocked
sensor may interrupt a process. If the operator can resolve the issue quickly,
the loss is defined as a minor stop instead of a breakdown, which completely
halts production.
4. Reduced speed: An operator may choose to run an instrument at a
processing speed lower than the instrument’s potential because of quality or
safety concerns associated with the higher speed. The reduced speed also
reduces the number of jobs the instrument can complete.
5. Quality defects and rework: The process may generate jobs that do not
meet the required specifications. In some processes, the operator can salvage
the job, but that requires extra time and effort. Anything that does not come
out correct on the first try counts as a defect.
6. Startup and yield: Processes may undergo a phase during startup when
jobs do not meet specifications. For example, if the instrument generates
faulty jobs as it warms up, those jobs count as startup or yield loss.
The first two losses – (1) breakdown and (2) setup and adjustment – affect
Availability. Similarly, (3) idling and minor stoppage and (4) reduced speed detract
from Performance. Finally, (5) quality defects and rework and (6) startup and yield
determine Quality. As discussed above, other reasons for delay, such as a shortage of
parts, can be classified as one of these types of losses.
26
Operator ownership of maintenance: TPM promotes the idea that when operators
gain responsibility and control of maintenance procedures, they perform maintenance
more effectively. If every mechanical error feels like an embarrassment, the operators
will go to greater lengths to prevent that error from occurring again. This greater
ownership occurs in the context of an Autonomous Maintenance (AM) program,
which leads operators to modify the work environment, the instrument, and the
maintenance procedures to drive towards zero defects. In the context of this project,
instrument modifications were not permitted under CLIA regulations. Chapter 7
discusses the specific activities of the AM team in greater detail.
Improved operator skillset: Tracing a defect to its true root cause requires a deep
understanding of the instrument. Therefore TPM involves extensive operator
training, most of it hands-on and self-driven. For example, operators are instructed
to touch every single component of the instrument and record any questions they
have about the instrument. The team records the questions and incorporates the
answers into training documents in the form of One-Point Lessons, which are
discussed in greater detail in Chapter 7.
Maintenance prevention and early equipment maintenance: The maintenance
performed during the functional lifecycle of an instrument captures only a part of the
total maintenance involved. In the manufacturing context in which TPM was
developed, instruments usually experience a high rate of defects immediately after
installation compared to the rate during normal functioning. This phenomenon arises
because the operators must determine all of the necessary operating parameters,
identify any imperfectly installed components, and integrate the instrument with the
rest of the operating environment. TPM includes early equipment maintenance,
which focuses on stabilizing the instrument as quickly as possible after installation,
thereby reducing the number of defects at the beginning of an instrument’s life.
In a medical context, start-up costs are usually associated with the extended
validation process required to confirm that the instrument operates according to
specifications and provides accurate results. Early equipment maintenance in this
context would focus on guaranteeing that the instrument requires the minimum
number of validation tests before starting patient testing.
Looking farther upstream in the instrument lifecycle, TPM also includes
maintenance prevention, discussed in Section 3.1, which aims to modify the
instrument design to eliminate maintenance steps completely. This component of
27
TPM requires collaboration with the instrument manufacturer and therefore often
falls outside the scope of what a company can directly accomplish in a TPM
program. This is especially true of a regulated industry like healthcare, where design
changes can be prohibitively expensive because of the cost of gaining regulatory
approval.
3.3 Drivers of success and failure Companies that effectively implement TPM show dramatic improvements in the
common measurements of plant performance – safety, quality, delivery, inventory,
and productivity – as well as employee morale. However, many organizations that
begin a TPM program do not succeed. The path from initial TPM activity to
significant results usually takes three to five years, and organizations often fail to
sustain TPM activities throughout that time [4].
The primary barriers to successful implementation are as follows [7]:
• Behavioral: Employees and/or management do not maintain the commitment,
involvement, and support to sustain TPM.
• Human and cultural: The company culture does not promote the individual
empowerment required in TPM or limits employees’ abilities to adapt to a new
approach to maintenance.
• Strategic: A lack of clear organizational goals or a poor framework for generating
decisions leads to widespread frustration and disillusionment with TPM.
• Operational: Poor organization around TPM tools, such as a lack of standard
operating procedures or a chaotic work environment, can interfere with the main
work of TPM, draining energy from the program.
• Technical: Employees do not gain a complete understanding of the TPM
principles and therefore are unable to effectively use TPM tools.
Of these, the behavioral barriers are the most commonly cited cause for TPM failure
[8]. Specifically, when top management does not maintain consistent support of TPM
throughout the implementation phase, employees will quickly return to the status
quo. The next largest barriers are organizational: when TPM is applied
inconsistently throughout an organization, the uneven application creates friction
28
that eventually causes the initiative to grind to a halt. The least important is
technical [7]. With a basic understanding of the principles of TPM and a clear,
coherent vision, an organization can succeed in implementing TPM. This ranking
agrees with the experiences of the author in trying to organize an autonomous
maintenance pilot project, as discussed in Chapter 7.
3.4 Chapter summary Approaches to maintenance range from the highly reactive Breakdown Maintenance,
which occurs in response to a system failure, to the proactive and holistic Total
Productive Maintenance, which aims to prevent breakdowns and streamline
maintenance procedures. An effective TPM program has the combined benefits of
increasing operator engagement, decreasing the frequency of instrument breakdowns,
and improving the rate of correctly performed jobs. However, TPM is notoriously
difficult to implement because it requires a significant cultural shift, which
companies struggle to maintain without the appropriate employee or management
support. Chapter 7 discusses the process of implementing Autonomous
Maintenance, a component of TPM, and the challenges associated with that pilot
project in this historical context.
In order to understand how Autonomous Maintenance applies here, we must first
understand the mechanics of the diagnostic testing platform that serves as the focus
of this thesis. The following chapter reviews the major components of the instrument
in detail and highlights the primary causes for mechanical errors, which an effective
maintenance program must address.
29
Chapter 4 Process description Having established the relevant company context and history of maintenance, we
now turn to the diagnostic testing platform that serves as the focus for this thesis. In
this chapter, we briefly discuss the analytical principle of the test, which involves
combining the patient sample with the appropriate reagents, creating a luminescence
reaction that allows the instrument to measure the concentration or identify the
presence of the compound of interest. In order to perform this sequence of automated
functions, the instrument contains several mechanical subsystems, of which 10 prove
critical to understanding the testing process. Each subsystem can lead to mechanical
errors that interrupt normal sample processing, wasting the reagent consumed during
the test and taking away from the time available for sample processing. We discuss
the leading causes of mechanical errors within each subsystem, their underlying
cause, and the routine maintenance procedures intended to reduce their occurrence.
It is also important to understand the organizational structures that interact with
the instrument. This chapter concludes by discussing the main stakeholders and how
their interests inform the resources available for this project.
4.1 Description of instrument
4.1.1 Analytical principle
The function of the instrument is to determine the presence of a target compound in
the sample. At a chemical level, the process generally proceeds as follows. The
sample is combined with a test-specific reagent in a test-specific reaction well and
incubated to allow the reagent to react with any of the target compound present in
the sample. Depending on the test, the target compound may bind to compounds
embedded in the well walls so that rinsing the well removes any unbound chemicals.
A second generic reagent provides a catalyst so that the bound compound
luminesces. By measuring the amount of light emitted, the instrument can measure
the amount of target compound present. The sample is considered reactive (the
compound is present) above a threshold high level of luminescence; non-reactive (the
compound is not present) below a threshold low level; or borderline (additional
30
testing may be required to determine the presence of the compound) between the
high and low levels.
4.1.2 Main mechanical systems
The instrument performs the sequence of tasks described in the previous section
using a sophisticated automated system that consists of the subsystems depicted in
Figure 4.1.
Figure 4.1: Map of main mechanical systems in instrument
(a) View from top of the instrument
(b) View from front of instrument
The mechanical subsystems interact with the patient sample as follows, with the
sequence of events involved in sample processing also shown in Figure 4.2:
i. Sample loading area: The operator loads samples through this portal.
ii. Reagent supply area: The operator loads reagent kits consisting of the liquid
reagents and the associated reaction wells into this refrigerated chamber on
the instrument. As soon as the instrument determines the type of test
Well Shuttle
Incubator
Wash Reagent
Proboscis
Signal Reagent
Proboscis
Luminometer
Reagent Supply Area
Sample Supply Area
Reagent Proboscis
Sample Proboscis
êFront
Wash Reagent
Proboscis
Signal Reagent
Proboscis Sample
Supply Area
Solid Waste Container
Incubator Reagent Supply
Area
Liquid Waste
Container
éTop
31
scheduled for the sample in question, it dispenses a reaction well into the
incubator through the well shuttle.
iii. Sample proboscis: The sample proboscis first picks up a disposable plastic
cone with the large end creating a seal around the body of the proboscis and
with a hole in the small end to allow sample to enter. By using the disposable
tip, the instrument minimizes the risk of cross-contamination between
samples. With the tip, the proboscis aspirates a specific volume of the sample,
as determined by the test to be performed, and dispenses the sample into the
reaction well dispensed in the previous step. The proboscis then discards the
tip in the solid waste container (see solid waste container description below).
iv. Reagent proboscis: The reagent supply area presents the appropriate type of
reagent for the test in question. Then, following the same process as the
sample proboscis, the reagent proboscis picks up a new plastic tip, aspirates a
specific volume of reagent, dispenses it into the reaction well with the sample,
and ejects the tip into the solid waste container.
v. Incubator: While the reagents react with the samples, the reaction wells sit
for a fixed amount of time as dictated by the specific test in the incubator,
which maintains the wells at body temperature. The incubator also transports
the well to the other subsystems involved in sample processing.
vi. Wash reagent proboscis: Following incubation, this proboscis dispenses the
wash reagent that removes any material not bound to the sides of the reaction
wells and removes the residual liquid.
vii. Liquid waste container: The residual liquid from the wash step is deposited in
a liquid waste container, which the operator empties regularly.
viii. Signal reagent proboscis: After the wash step, this proboscis dispenses the
reagent that catalyzes the luminescence reaction.
ix. Luminometer: The luminometer detects the amount of luminescence in the
sample relative to an internal reference.
x. Solid waste container: Finally, the instrument deposits the used reaction well
in this waste container, which the operator empties regularly. This container
is also used for plastic tips used on the reagent and sample proboscises.
32
Figure 4.2: Path of sample processing within instrument
4.1.3 Mechanical functions and common errors
This instrument was selected as the target for this project because of the relatively
high fraction of reagent waste that the testing platform generated over the previous
year. This section will discuss the major mechanical function of the main subsystems
and the dominant types of mechanical errors observed in each subsystem, each of
which lead to reagent waste. Whenever a mechanical error occurs, the instrument
attempts to re-run the test on that sample. If the instrument cannot automatically
schedule a repeat test, the operator can manually re-run the test at a later point.
The instrument reports an error code when sample testing is interrupted so that the
operator can determine the cause of the error.
i. Sample loading area: This component involves (1) a rotating set of positions
where samples can be loaded and (2) sensors that read sample barcodes in
order to determine which test or tests have been ordered for each sample. The
mechanics of this subsystem are relatively robust and are only affected if gross
contamination clogs the movement of the rotating components or blocks the
sensors.
ii. Reagent supply area: Reagent kits consist of containers of the liquid reagent
and sleeves containing nested stacks of the reaction wells. The reagents do not
require any advanced preparation, and the lifespan of a single reagent pack on
the instrument is long compared to the rate of reagent consumption, so
reagent quality concerns or expiration rarely generate waste at Quest
Diagnostics labs. However, if the reagent kits are agitated before being loaded
onto the instrument, bubbles at the top of the reagent container can cause
issues for the reagent proboscis.
1. Operator loads sample into Sample Supply Area. 2. Reagent Supply dispenses reaction well to incubator through
Well Shuttle. 3. Sample Proboscis aspirates patient sample. 4. Sample Proboscis dispenses patient sample into reaction well. 5. Reagent Proboscis aspirates reagent in Reagent Supply Area. 6. Reagent Proboscis dispenses reagent into reaction well. 7. Reaction well stays in Incubator for designated length of time. 8. Wash Reagent Proboscis rinses residual liquid from reaction
well. 9. Signal Reagent Proboscis dispenses signal reagent into
reaction well. 10. Luminometer measures luminescence in reaction well. 11. Incubator ejects reaction well into Solid Waste Container.
1
2 3 4
5 6
89
10
11
33
The more common mechanical error for the reagent supply area relates to the
reaction wells. The reagent kits hold the wells in a sleeve open at the bottom
and top. A screw with a tip fitted to the diameter of the sleeve moves down a
fixed number of mechanical “steps” while pushing on the top well to cause a
single well to fall out the bottom of the sleeve into the incubator. A sensor
within the incubator confirms that a single well has been dispensed. Moisture
in the instrument can lead the wells to stick together rather than nesting
loosely, preventing a well from dropping or causing two wells to drop
simultaneously. In each instance, the instrument will abort the test and
discard any wells in the well shuttle into the solid waste container.
iii. Sample proboscis: The sample proboscis consists of an external sleeve and an
internal hollow piston covered with a plastic cap. The proboscis is connected
to a pump that draws air through the hollow piston in order to aspirate and
dispense sample liquid. The piston cap protects the narrow piston from
exposure to liquid, and it creates small openings between the sleeve and the
piston through which air must pass before it can be drawn up through the
piston. During use, a disposable plastic tip fits onto the external sleeve. In
this way, sample only comes into contact with the disposable tip rather than
the piston, piston cap, or external sleeve.
The main mechanical issues from the sample proboscis derive from debris
accumulating on the proboscis that compromises airflow. As the proboscis
moves within the instrument, sample can splash up onto the piston cap and
dry, leaving residue. The instrument software determines aspirated and
dispensed liquid volume by measuring the pressure changes detected at the
pump. Therefore if the debris disrupts airflow, it can lead to deviations from
the expected pressure profile. Then, because the instrument cannot guarantee
the accuracy of the volume measurement, it abandons the test.
Sample integrity issues can create the same effect. For example, insufficient
sample volume, heterogeneity in sample consistency, or bubbles in or on the
sample can also lead to unusual pressure profiles, causing the instrument to
stop processing the sample.
Mechanical errors also occur from interactions with the plastic tips: the
proboscis may fail to pick up the tip on the first try, it may fail to eject the
tip after processing, or the tip may have smaller than usual opening, impeding
airflow.
34
iv. Reagent proboscis: The function and primary error types of the reagent
proboscis are identical to those of the sample proboscis. The reagent proboscis
generally aspirates and dispenses a greater amount of liquid than does the
sample proboscis, so debris may accumulate more rapidly than in the sample
proboscis.
v. Incubator: The incubator is a heated compartment containing several
concentric rings with holes where the reaction wells sit. The rings rotate the
wells among positions dedicated for different steps in the process. The
compartment has holes through which sample and the different types of
reagent are dispensed so that the well remains inside the incubator through
these steps.
The wells also are moved between rings for different stages of the incubation.
To move the well between rings, a small metal rod pushes up through the
holes in the ring to lift the well out of its position and into a shuttle. Inside
the shuttle, metal tabs grip the well until the shuttle moves the well into
position over the appropriate ring, at which point the tabs release. This
process also repeats to allow the instrument to measure the luminescence: the
rod lifts the well into the shuttle, which holds the well in position for
measurement and then transports the well to the disposal chute leading to the
solid waste container.
Thus the incubator involves many moving parts that must coordinate
precisely in order to process samples. If any of these movements breaks down,
the incubator can jam, which interrupts all of the tests in the incubator at the
time – as many as 100. Mechanical jams can occur when sample or reagent
splashes out of the wells onto mechanical components; when a proboscis is out
of alignment or is leaking, causing liquids to fall onto mechanical components;
when debris prevents the many position sensors from verifying the location of
moving components; when fibers are left behind after maintenance activities;
when a previous jam in the well transport system causes a well to get caught
in moving components and shatter, leaving behind plastic chips; and when the
solid waste container overflows and backs up into the incubator.
vi. Wash reagent proboscis: The wash reagent proboscis consists of two hollow
metal straws, one of which dispenses wash reagent and one of which aspirates
the mixture for transport to the liquid waste container. The design is simpler,
35
and the wash reagent is both more dilute and more homogeneous than
samples or the reactive reagent. As a result, wash reagent proboscis does not
usually experience the same problems with blockage seen in the reagent and
sample proboscises. Instead, the primary mechanical issue facing the wash
reagent proboscis is the integrity of the fluidics system. If a connection leaks
or a crack appears in a line, then air bubbles can infiltrate the lines, leading
to leakage. If the instrument detects that the incorrect amount was dispensed,
then it will abort the test in progress. In addition, leakage that falls into the
incubator can contribute to mechanical jams.
vii. Liquid waste container: The liquid waste container collects the liquid waste
from sample processing and does not involve a high level of mechanical
complexity. The primary problem arising from this subsystem occurs when
the waste container fills up during sample processing, which will cause the
instrument to stop processing additional samples.
viii. Signal reagent proboscis: The signal reagent proboscis consists of two metal
straws that dispense the two possible types of signal reagent. Like the wash
reagent proboscis, the primary source of mechanical errors with this
subsystem lies in air intrusions into the fluidics system.
ix. Luminometer: To generate accurate results, the luminometer requires
complete exclusion of outside light. The instrument isolates the luminometer
by performing the measurement in a closed compartment within the
incubator. However, if the compartment cover is not seated properly, external
light sources may infiltrate the chamber.
x. Solid waste container: Similar to the liquid waste container, the solid waste
container involves little mechanical complexity. However, if not emptied
regularly, the wells can accumulate, back up into the disposal chute, and
eventually jam the moving components of the incubator.
4.1.4 Reagent waste
The instrument performs the process described above for three types of samples:
patient samples, quality control samples, and calibration samples. Quality control
materials are used to confirm that the instrument is continuing to generate
consistent results over time, as described in greater detail in Chapter 6. Calibration
samples are used to fix the instrument’s luminescence readings to known values,
36
thereby correcting for any drift that happens over time or to accommodate for any
differences in reagents that would lead to subtly different results.
The outcome of all of these tests will either be a reactive, nonreactive, borderline, or
incomplete result. The reactive and nonreactive tests indicate the presence or
absence of the target compound, as discussed in Section 4.1.1. A borderline result
indicates a luminescence level that does not strongly indicate either the presence or
the absence of the target compound and triggers a repeat test to confirm its
reactivity. An incomplete result arises when a mechanical error interrupts normal
sample processing. In this instance, a well dispensed by the instrument does not
generate a luminescence reading.
According to the definitions in this analysis, reagent is considered waste when it does
not lead to reactive or nonreactive results for patient samples. Thus quality control
and calibration samples are counted as waste for the purposes of this analysis, even
though they are critical components of a quality program and are required by CLIA
and Quest SOP’s. Likewise, borderline and incomplete results are both considered
waste. However, because borderline results arise from the chemical state of the
patient sample, they were not a focus of this analysis.
The platform of focus for this study was selected because preliminary data indicated
that it generated a relatively high fraction of waste and, anecdotally, it experienced a
relatively high rate of mechanical errors.
4.2 Maintenance overview In addition to running patient, quality control, and calibration samples as part of
normal operations, instrument operators also perform routine preventive
maintenance and troubleshooting. When preventive or reactive maintenance goes
beyond the training of operators, representatives from the instrument manufacturer
also perform service. This section discusses the types of maintenance performed by
each group.
4.2.1. Operator maintenance
The operators clean the major instrument subsystems at a frequency recommended
by the manufacturer and determined by the approximate rate of debris accumulation
on that subsystem. Cleaning includes all proboscis tips; all moving components of
the incubator, including the metal rods that lift and lower reaction wells; and
surfaces where samples might spill, such as the sample supply area. After manually
37
cleaning the proboscises, the operator also runs an instrument program that confirms
effective function of all proboscises.
In addition to cleaning, operators must also maintain adequate supplies of
consumables like the plastic tips for the reagent and sample proboscises and the
reagent kits, and they must regularly empty the solid and liquid waste containers.
The testing platform includes software that guides the operator through all the steps
involved in maintenance with detailed instructions of the cleaning supplies and
procedures and diagrams of the targets of cleaning.
Troubleshooting usually involves replicating a set of routine maintenance activities
to address the potential source of debris. The operators also have the authority to
replace certain parts, such as the reagent and sample proboscises. The user interface
of the testing platform allows an operator to look up the recommended
troubleshooting process for any condition code that the system generates. Beyond a
limited set of activities, a vendor representative must intervene.
4.2.2. Vendor maintenance
Engineering representatives from the manufacturer perform more complex
troubleshooting operations and more comprehensive preventive maintenance
activities. They may replace components such as connectors and fluid lines for the
signal reagent system.
4.3 Stakeholder analysis
4.3.1. Primary groups and their interests
Despite this project’s narrow focus – reagent waste on a single testing platform – it
nonetheless affected many groups within Quest Diagnostics at both a local and a
national scale. Aligning the interests of these overlapping groups proved a leading
challenge throughout the project. This section describes the main groups, their
interests, and the leading sources of friction.
i. Operators: The instrument operators prioritized completing their work
effectively and preserving quality of patient testing results. Beyond
successfully completing their work, they also expressed a desire to minimize
the frustrations involved in their work, such as the hassle of repeating dozens
of patient samples because of a mechanical error.
38
ii. Laboratory management: The supervisors and managers at a local level
prioritize testing quality as well as general laboratory metrics. Result
turnaround time and employee productivity (e.g., tests per full-time
employee) received the most attention during the thesis.
iii. Regional management: Higher tiers of laboratory management set the
performance goals for the laboratory within the larger context of the
Invigorate initiative. They therefore placed more value on long-term
investments like training and improvement initiatives like Total Productive
Maintenance than local managers and supervisors did.
iv. Manufacturer: The manufacturer of the testing platform had three main
interests. First, they wanted to strengthen the relationship with Quest
Diagnostics to drive more business in the future. Second, they wanted to
preserve their internal norms related to work-life balance. They resisted
requests that significantly increased the workload of any one employee and
argued for longer timelines for delivering on requests for access to data or
information about instrument performance. Third, reagent serves as a
significant source of revenue for the manufacturer, so they will suffer in the
short term from any process improvements that decrease the amount of
reagents that Quest Diagnostics purchases. This short-term impact is
balanced against the vendor’s drive to become a strategic business partner for
Quest Diagnostics, thus improving the likelihood of future revenue
opportunities. While this issue did not arise explicitly during the thesis, it
served as an important backdrop for understanding the manufacturer’s motivations.
v. Best Practices Team (BPT): The national BPT that oversaw the standard
operating procedures (SOPs) for the target testing platform wanted primarily
to establish guidelines that would guarantee the quality of test results. They
acknowledged the value from reducing reagent waste but resisted any actions
that could potentially reduce result quality.
vi. Reagent waste team: The national working group tasked with generating
savings within the Invigorate program focused on savings opportunities that
would not compromise result quality.
39
4.3.2. Potential conflicts
The main conflicts that arose among these groups are the tensions between (1)
productivity goals and laboratory improvement initiatives, (2) reagent costs from
excessive quality control testing and benefits from more conservative quality
measures, and (3) manufacturer intentions and capabilities. Each conflict is discussed
briefly below.
i. Productivity goals vs. laboratory improvement initiatives: The regional
management championed improvement activities such as Total Productive
Maintenance. However, implementing such activities requires a significant
investment of equipment and operator time. While laboratory management
supported the premise of these improvement initiatives, they were constrained
by conflicting requirements related to employee productivity and turnaround
time, which limited their ability to dedicate staff to the thesis work.
ii. Reagent costs from excessive quality control testing vs. benefits from more
conservative quality measures: Quality control is a critical and unavoidable
part of testing. Applying quality control practices that increase the frequency
or quantity of quality control material consumed may add certainty to the
results, but it always increases reagent costs. The reagent waste team
identified a few opportunities to reduce reagent costs that involved adopting a
less conservative quality control policy. Because the BPT prioritizes quality so
highly, these suggestions sparked significant resistance, regardless of whether
data indicated that using the more conservative approach actually resulted in
higher quality.
iii. Manufacturer intentions vs. capabilities
Finally, the manufacturer intended to use this thesis as an opportunity to
build a stronger relationship with Quest Diagnostics. However, the analysis
revealed gaps in their capabilities, such as a lack of data correlating an
interrupted test to a specific condition code, that could not be resolved on the
timescale of the thesis.
4.4 Chapter summary The target instrument for this thesis involves 10 major mechanical subsystems
that must function in concert to generate patient results. A mechanical error at
any one of the subsystems has the potential to interrupt sample processing,
forcing the instrument to discard the sample, which then must be retested.
40
Certain types of errors, including most errors in the incubator, can interrupt
sample processing for all tests in progress at the time – as many as 100. Routine
operator maintenance and vendor maintenance activities are designed with the
intention of minimizing the occurrence of these errors.
Any changes to current procedures must balance the interests of the primary
groups involved with the instrument. The balance between productivity goals and
laboratory improvement activities constrains the activities of the Autonomous
Maintenance team, as discussed in Chapter 7. The balance between reagent
costs from excessive quality control and the benefits from more conservative
quality control measures informs the recommendations about quality control
procedures discussed in Chapter 6. Finally, the tension between the
manufacturer’s intentions and their capabilities limited the possible depth of
quantitative analysis in this thesis. The next chapter discusses the current state
of the process, given the available data.
41
Chapter 5 Current state analysis Building on the process details discussed in the prior chapter, this chapter discusses
the current state of the process as of the beginning of the project. Establishing the
current state involves process observation and data analysis. Observed processes
include standard operation, quality control, routine maintenance, and
troubleshooting. Observation occurred at four laboratories in distinct geographical
areas of the United States to understand the basis for variations in preliminary data
about reagent waste. For the purposes of the following discussion, the term “waste” refers to any reagent that does not contribute to a unique patient result. Therefore
reagent used in quality control and calibration – necessary and legally mandated
parts of a quality system – is termed waste, despite their role in the process. When
observations of quality control procedures revealed variations in implementation, all
sites using the testing platform were surveyed to gather comprehensive information
about quality control practices.
In parallel with direct observations, the process was also analyzed using three forms
of instrument data: (1) usage counters, which track how many wells generate
patient, quality control, calibration, or incomplete results; (2) condition code reports,
which record the frequency of mechanical errors on the instrument; and (3)
incubator maintenance charts, which show how long the incubator was opened for
maintenance, thus providing a crude metric for maintenance adequacy. The
preliminary observations from the data analysis include:
1. Quality control is a bigger source of waste than mechanical errors
2. Three instrument subsystems drive the majority of mechanical errors
3. The most common condition codes may be addressed through
improved maintenance.
Thus quality control and mechanical errors arise as the major sources of waste to be
tackled in this thesis. Chapter 6 goes on to discuss how to address the variations in
42
SOP interpretations, and Chapter 7 discusses how to unify maintenance and
operation around national best practices.
5.1 Process observation Observations of the process at four laboratories across Quest Diagnostics led to a
preliminary assessment of some of the drivers of reagent waste, specifically,
individual laboratories exceeding quality control requirements and a lack of
consistent dissemination of best practices in maintenance and operation.
5.1.1 Activities completed
This section discusses in detail the types of observations performed and the content
of the survey of quality control practices.
5.1.1.1 Observed components of routine operation
As described in Chapter 4, the interactions between the operators and the testing
platform consisted of (i) routine operation, (ii) quality control, (iii) routine
maintenance, (iv) troubleshooting, and (v) calibration. As discussed below in
Section 5.2.2, calibration comprised a small fraction of overall reagent usage, so
calibration was not a focus during the observations.
Steps (i)-(iv) were observed as described below in detail. The sections are separated
into processes that could be observed directly and those that required conversations
with the operators, lab manager, or other employees to evaluate.
A critical component of the analysis involves comparisons across different sites.
Anecdotally different sites experienced dramatically varying rates of reagent waste.
To understand the basis for these differences, observation was repeated at four
laboratories in distinct geographical regions: Marlborough, MA; Wood Dale, IL;
West Hills, CA; and Miramar, FL.
i. Routine operation
- Observed:
• Routine operations starting from delivery of patient samples to the
testing area, continuing through sample processing and result
interpretation, and concluding with returning samples to relevant
storage; and
• The inventory policy by noting the location and quantities of
consumable materials.
43
- Discussed:
• The general sample mix and the turnaround time requirements for each
lab;
• The layout of the lab and any recent or anticipated changes to the
number, generation, and organization of the instruments;
• The operator’s experience of running the instruments and qualitative
impressions of the main sources of waste; and
• The occurrence of reagent expiration, either while in storage or while
loaded on the instrument.
• Operators’ experience of reagent expiration.
ii. Quality control
- Observed:
• The containers used for quality control; and
• The location and storage approach for quality control materials.
- Discussed:
• The frequency of quality control samples;
• The acceptance criteria for quality control results;
• Strategy for addressing quality control failures; and
• Relevant lab history leading to quality control policies.
iii. Routine maintenance
- Observed all weekly maintenance happening within a one-week timescale,
including removing waste, replacing consumables, and cleaning instrument
components.
- Discussed:
• Training history of operator performing maintenance; and
• Operator’s understanding of basis for maintenance procedures.
iv. Troubleshooting:
- Observed operators troubleshooting mechanical errors on the instrument when
the opportunity arose during routine operation.
- Discussed what the operators perceived to be the most common reasons for
interruptions to sample processing and their strategies for addressing the
issues.
44
In addition to these structured observations, informal conversations were held with
key laboratory personnel to understand the main challenges facing that laboratory,
relevant process improvement initiatives, and unique qualities of the client base that
influence operations.
5.1.1.2 Prepared quality control survey
The laboratory observation highlighted variations in how laboratories interpreted the
SOPs for the testing platform. In order to understand the full range of quality
control practices for this instrument, a survey was designed with input from the
BPT and circulated to all laboratories using the instrument.
The questions within the survey included the following:
1. How often does your lab run positive quality control materials?
2. How often does your lab run negative quality control materials?
3. What size container does your lab use for quality control materials?
4. Which statement applies most closely to your lab’s practices:
a. “We prepare new containers of quality control material for each
use.”
b. “We prepare a container of quality control material and reuse it for
multiple rounds of quality control testing, but we do NOT replenish
the container with new quality control material.”
c. “We prepare a container of quality control material and reuse it for
multiple rounds of quality control testing, and we replenish the
container from the same lot of parent quality control material as
needed.” 5. Does your lab refrigerate quality control material between uses?
6. Does your lab cover the container of quality control material between
uses? If so, what kind do you use?
7. For qualitative assays, does your lab track quality control results
qualitatively or quantitatively?
8. How does your lab respond when a mechanical error occurs on a
quality control sample, leading to an incomplete result?
The responses to the survey are discussed in Chapter 6.
45
5.1.2 Preliminary observations
Preliminary observations from visiting the four laboratories can be summarized as
follows:
1. Mechanical errors are a major source of irritation for operators;
2. Operator knowledge levels about instrument function, operation, and
maintenance varied within and across labs;
3. Individual operators/labs develop local process improvements, but do
not consistently share with other labs; and
4. Interpretation of quality control requirements in SOPs varied across
labs.
i. Mechanical errors are major source of irritation for operators
Operators at all sites perceived the instrument to be sensitive to mechanical
errors, which fall into two general categories: transient errors and shutdown
errors. For the purposes of this thesis, transient errors are defined as mechanical
errors that affect only the sample currently being processed, and shutdown errors
are those that affect all samples being processed by the machine at that time
(i.e., including those in the incubator). In all cases of a mechanical error, the test
is re-run until a result can be delivered to the requesting provider.
A common example of a shutdown error is an incubator crash. The incubator
contains many moving mechanical parts, and if a jam occurs at any point in the
incubator, the instrument rarely can recover without operator intervention. At
that point, all sample processing halts, and the operator must remove the jam
before the machine can continue processing. The instrument may have up to 100
samples in the incubator at one time, and all these are forfeited because of the
potential for contamination when the operator opens the instrument to address
the jam.
An example of a transient error occurs when the opening for airflow in the
reagent proboscis becomes slightly occluded. The blockage may occasionally
interfere with airflow through the proboscis and therefore with the instrument’s ability to calculate the quantity of reagent aspirated or dispensed. When the
instrument cannot calculate this quantity to a high degree of certainty, it will
jettison the reaction currently being processed and try again with a new aliquot
of sample. This error may occur periodically until the proboscis is cleaned during
routine maintenance, but sample processing will otherwise continue without
interruption.
46
There are two addenda to the definitions as outlined. First, a transient error can
also escalate into a shutdown error. In the example of a reagent proboscis above,
a more serious blockage can cause sequential errors of the same type. When such
an error repeats for a threshold number of times, the instrument stops all sample
processing until the issue is resolved, usually through operator maintenance.
Second, a shutdown error does not always lead to the loss of all samples in
progress. If the shutdown error derives from an issue with the reagent proboscis,
the incubator does not need to be opened, so there is no potential for
contamination. In this instance, the only tests that must be discarded are those
that incubate longer than the time established by the standard operating
procedure and therefore may have invalid results. If the operator resolves the
problem quickly, few if any tests may be discarded.
As a rule, the instrument discards tests whenever there is a mechanical concern
for contamination or incorrect results, per standard industry practice.
Operators perceive the instruments to be sensitive to mechanical errors because
of the impact of shutdown errors. While transient errors generate more reagent
waste than shutdown errors, the shutdown errors have an outsized effect on the
operators because shutdown errors must be addressed immediately, creating
disruptions to the operator’s regular schedule; and increase the effective workload
for a given shift, increasing the difficulty of meeting turnaround time
requirements.
However, operators generally acknowledged that the current instrument
experiences fewer mechanical issues than the previous generation of the
instrument, which Quest Diagnostics is phasing out of operation.
ii. Operator knowledge levels about instrument function, operation, and
maintenance varied within and across labs
The level of operator skill varied within labs. Generally skill and depth of
knowledge increased with experience, and operators who had attended a weeklong
training session hosted by the instrument vendor had considerable expertise. The
experts were not necessarily responsible for routine maintenance and operation: in
order to provide development opportunities and to encourage workforce
flexibility, Quest Diagnostics promotes cross-training, so the experts were usually
responsible for training their colleagues on the instrument so that more people
were available to operate or maintain the instrument.
47
The skill levels of some of the less experienced operators suggested that there was
an opportunity to strengthen and unify training procedures, especially as they
related to maintenance. For example, one operator spent time cleaning a
component that, because of the types of testing performed in Quest Diagnostics
laboratories, is never used and therefore has no impact on instrument
performance.
iii. Individual operators/labs develop local process improvements, but do not
consistently share with other labs
Individual labs developed innovative ways to improve operation and maintenance
of the instrument. Encountering ergonomic stress when removing debris from a
deep shelf on the instrument, one laboratory created a tool that helped sweep off
debris and eliminate the strain. The same lab had collaborated with the vendor
to apply labels to key components within the instrument with helpful reminders
about proper operation and maintenance (e.g., “Wait until light turns green
before opening compartment”). A different lab designated a cart to hold
maintenance equipment for the instrument to reduce the time spent collecting the
necessary tools every day. Each of these ideas was developed locally and was not
shared outside that laboratory even though other labs would benefit from those
insights.
iv. Interpretation of quality control requirements in SOPs varied across labs
All of the labs met the minimum quality control requirements as specified by
SOPs; however, the precise interpretations varied across labs, with some labs
choosing to exceed requirements significantly. The variations relevant for this
study in waste reduction were as follows:
1. Frequency: some labs ran quality control samples more often than was
required by the SOP.
2. Container size: the size of the container determines the amount of
“dead volume”, i.e., liquid the instrument cannot access, that must be
discarded after testing.
3. Acceptance criteria: some tests on the target platform yielded
qualitative results and others yielded quantitative results. Laboratories
differed in the acceptance criteria for quality control samples, with
48
some tracking all tests through quantitative methods, and others using
a qualitative or quantitative methods depending on the type of test.
4. Reuse policy: one lab prepared new containers of quality control
material for every use, while another used a single container of quality
control material for multiple batches. Yet another maintained a
container of quality control material that was replenished periodically
with quality control material from the original parent batch.
5. Storage approach: reusing quality control materials creates the
possibility of evaporation of quality control material between batches.
Some labs opted to cap and refrigerate the quality control containers in
between batches while others did not cap or refrigerate.
6. Approach to mechanical errors on quality control samples: occasionally
a transient error would occur on a quality control sample. One lab
espoused the policy of rerunning all samples from the affected assay
when this occurred. A different lab repeated several patient samples
and treated the results as sufficient quality control if they agreed with
the results as reported previously.
5.2 Data collection and evaluation
5.2.1 Sources of data
The qualitative observations described in Section 5.1 highlighted the need to
quantify the sources of waste and measure the impact of mechanical errors on
reagent usage. The data sources used to generate the results discussed in Chapters
6 and 7 consisted of usage counters, condition code reports, and incubator
maintenance charts.
i. Usage counters
The instrument vendor tracked the outcome of every well dispensed by each copy of
the testing platform at Quest Diagnostics. Outcomes fell into four categories:
1. Patient – completed tests of patient samples;
2. Quality control – completed tests of quality control samples;
3. Calibration – completed tests performed during instrument calibration;
and
49
4. Incomplete – any of the three previous types of samples that were
interrupted during processing because of a mechanical error.
Because of the relatively infrequent communication between the instruments and the
vendor servers, usage counters could only be recorded on a weekly basis. They
therefore served to provide a general sense of large trends over time or across
business units but were not valuable in evaluating countermeasures tested on shorter
timescales.
The usage counters do not take into consideration the amount of quality control
material that remained in the sample container after testing and was discarded. We
were unable to measure this quantity directly, instead opting to calculate the
minimum required dead volume depending on sample container and reuse policy.
ii. Condition code reports
The usage data provides insight into the frequency of mechanical errors but not their
cause. To understand the drivers of mechanical errors, the vendor provided reports
of the condition codes generated by the instruments during sample processing.
Condition codes ranged in content and severity, and most condition codes did not
indicate that a test had been lost. When the instrument software linked a condition
code to a test in progress, it is assumed that the test was interrupted, leading to an
incomplete result. This correlation did not hold 100% of the time, so vendor software
specialists periodically reviewed the condition codes to confirm data accuracy.
We could not, however, overcome the other weakness of the condition code data:
those data alone did not indicate whether the condition code led to a transient error
or a shutdown error. Therefore the number of incomplete tests captured by the usage
data is expected to exceed the number of condition codes.
The vendor further summarized the condition code reports by sorting the codes into
subsystems. Most condition codes were linked to a physical subsystem within the
instrument such as the incubator, reagent proboscis, or sample proboscis. The
condition codes were sorted by relevant subsystem to highlight instrument
components that required more or different maintenance. As noted above, the
frequency of condition codes did not correlate exactly with the number of tests lost
because a shutdown error would result in one instance of a condition code but many
– up to 100 – lost tests.
50
Instrument documentation provided some insight into potential root causes for the
mechanical errors and troubleshooting steps. Instrument engineers from the vendor
also drew from their experience to suggest operator actions that could prevent the
most common codes.
iii. Incubator maintenance charts
The vendor was able to record automatically the time that the instrument incubator
was open. The vendor’s maintenance procedures for the incubator required at least
15 minutes of operator time, as verified by the vendor and confirmed independently
by a team within Quest Diagnostics, so these data provided a crude indication of the
quality of incubator maintenance. The data did not confirm that maintenance was
being performed correctly, but if the incubator were open for fewer than 15 minutes,
they would strongly indicate that maintenance was performed incorrectly.
Because of the relatively small amount of information provided by these charts, they
were not included in subsequent analyses. Instead, they were used as an illustrative
tool of the importance of complete maintenance during site observations.
5.2.2 Preliminary observations
Preliminary results from these data sources established the current conditions for the
target platform and guided the choice of countermeasures discussed in Chapters 6
and 7. The primary insights from these preliminary data were as follows:
1. Quality control is a bigger source of waste than mechanical errors
2. Three instrument subsystems drive the majority of mechanical errors
3. The most common condition codes may be addressed through
improved maintenance
i. Quality control is a bigger source of waste than mechanical errors
Recall that while quality control is a critical part of the patient testing process, it
qualifies as waste for this analysis because it is not a patient test result. Usage
counter data from March-May 2016 showed that within an example laboratory,
quality control tests accounted for 7.1% of all reagent usage at the site, almost twice
as much as incomplete tests from mechanical errors, which accounted for 3.6% of
reagent use, as shown in Table 5.1. Calibration tests accounted for a negligible
0.16% of reagent used, and there was no indication of excessive calibration or
frequent calibration failures that would prompt further investigation. Therefore
calibration is treated as an insignificant source of waste for the remainder of the
thesis.
51
Table 5.1: Reagent usage March-May
This table shows the fraction of reagent that generated four types of results: patient,
quality control, calibration, and incomplete. The incomplete results arise when a
mechanical error interrupts normal sample processing. The rates remain stable from
month to month, and the usage patterns at an example laboratory are consistent
with the overall picture within Quest Diagnostics. However, during this time period,
the example lab consistently used quality control at a slightly higher rate than
average, and mechanical errors occurred at a slightly lower rate.
Month / Laboratory Patient Quality Control Calibration Incomplete
Marc
h
Company Average 90.1% 5.3% 0.2% 4.5%
Example Lab 87.2% 7.7% 0.2% 4.9%
Apri
l Company Average 90.5% 5.2% 0.2% 4.1%
Example Lab 89.3% 6.9% 0.1% 3.7%
May Company Average 90.3% 5.1% 0.2% 4.4%
Example Lab 91.4% 6.3% 0.2% 2.1%
All
Company Average 90.3% 5.2% 0.2% 4.3%
Example Lab 89.1% 7.1% 0.2% 3.6%
Nationally over the same time period, quality control and incomplete tests accounted
for 5.2% and 4.3% of tests, respectively. The results for the example lab were largely
consistent with the company-wide averages over the same time period, indicating
that instrument behavior in the example laboratory is representative of normal
instrument operation, with two exceptions: the example laboratory consistently
performed slightly more quality control tests and experienced slightly fewer
mechanical errors than the national average. These subtle differences aside, the usage
data suggests that insights from activities in example might be applicable at a
national scale.
ii. Three instrument subsystems drive the majority of mechanical errors
During March-May 2016, prior to the start of the internship, three instrument
subsystems accounted for 89.7% of mechanical errors, based on the condition code
reports. As shown in Table 5.2, the sample proboscis, reagent supply area, and the
reagent proboscis accounted for 49.5%, 20.3%, and 19.9% of mechanical errors,
respectively. As discussed in Section 5.1.1 above, operators perceive incubator
crashes to generate the most waste, probably because of the outsized impact that
52
any one incubator crash can have, in terms of both lost tests and pressure on
operators. However, incubator crashes occur infrequently compared with transient
errors on the reagent proboscis, accounting for only 1.7% of the mechanical errors.
Table 5.2: Error code data for an example laboratory March-May
This table shows the relatively frequency of mechanical errors occurring on the three
instances of the testing platform in the example lab. The mechanical errors are
sorted by the subsystem where the error occurred. The “Other” subsystem includes
software complications or other errors that are not associated with a major
instrument subsystem.
Instrument
1 Instrument
2 Instrument
3 Total
Sample Proboscis 39.3% 67.4% 35.2% 49.5%
Reagent Supply 27.5% 13.9% 22.2% 20.3%
Reagent Proboscis 17.4% 12.9% 31.5% 19.9%
Luminometer 11.1% 0.6% 8.9% 6.1%
Other 2.6% 2.1% 0.9% 1.9%
Incubator 1.5% 2.4% 1.0% 1.7%
Sample Supply 0.7% 0.8% 0.3% 0.6%
iii. The most common condition codes may be addressed through improved
maintenance
Review of the condition codes for the reagent and sample proboscis with guidance
from vendor engineers revealed that the most common codes for these subsystems
can be addressed through improved maintenance. 90% and 54% of the codes for the
reagent proboscis and sample proboscis, respectively, arise because debris inhibits
smooth airflow through the proboscis, triggering transient errors and potential
shutdown errors. By increasing the intensity or frequency of maintenance on these
subsystems, the laboratory may reduce the frequency of these errors, thereby
reducing reagent waste and improving turnaround time.
Of the reagent supply errors, 74% came from a single condition code: two reaction
wells were dispensed into the incubator at once. This occurs primarily because
humidity or some other binding agent causes wells to stick together. Other than
ensuring that the reagent kits are stored in low-humidity conditions throughout the
chain of custody, there are no operator-based actions that can reduce the frequency
of this error. Therefore the reagent supply condition codes are not considered good
targets for maintenance-based countermeasures and are not discussed further in the
thesis. However, based on this result, the vendor began discussions with the reagent
53
kit manufacturer to test upstream process changes that could reduce the frequency
of these errors.
5.3 Chapter summary Routine operations including quality control, maintenance, and troubleshooting were
observed at four geographically separated laboratories, leading to the following
observations:
1. Mechanical errors are a major source of irritation for operators;
2. Operator knowledge levels varied within and across labs;
3. Individual operators/labs develop local process improvements, but do
not have a clear channel for sharing with other labs; and
4. Interpretation beyond the minimum quality control requirements in
standard operating procedure (SOPs) varied across labs.
Despite the first observation above, analysis of the usage data indicated that quality
control samples actually constitute a larger fraction of reagent use than mechanical
errors, i.e., they lead to more waste. This observation motivates the efforts discussed
in Chapter 6 to identify unnecessary quality control practices that lead to reagent
costs without commensurate benefits to the quality of results.
Nevertheless, mechanical errors occur in approximately 4.3% of tests. These errors
occur predominantly in three of the instrument subsystems described in Chapter 4:
the sample proboscis, the reagent proboscis, and the reagent supply. The most
common mechanical error associated with the reagent supply system usually arises
from manufacturing conditions, thus falling outside the scope of this thesis. However,
the leading errors for the other two systems may be preventable through improved
maintenance procedures that go above and beyond manufacturer requirements. This
observation motivates the Autonomous Maintenance team activities discussed in
Chapter 7 that intend to reduce instrument errors through enhanced maintenance.
54
Chapter 6 Countermeasures to eliminate quality control waste The previous chapter established that quality control was one of the two major
sources of reagent waste on the testing platform. In addition, laboratories varied
significantly in their interpretation of SOPs related to quality control procedures.
This chapter discusses the approaches used to investigate the source of these
variations and develop recommendations for aligning practices with the least
wasteful approach that meets quality control requirements.
The chapter begins with a summary of the relevant requirements for quality control
– both regulator-enforced and internally established – which set the foundation for
what these procedures must achieve. Then we review the results of the quality
control survey, which showed variations in quality control frequency, container size,
acceptance criteria, reuse policy, storage method, and approach to mechanical errors.
These variations led to large differences in how much quality control material was
consumed and how much reagent was used in quality control testing, both of which
qualify as waste.
The different approaches are then discussed in terms of their potential impact on
waste and result quality. Based on this analysis, one example laboratory opted to
reduce the frequency of negative quality control sample testing to be consistent with
the SOP, resulting in a reduction in reagent use in quality control testing by nearly
half.
This analysis reveals the importance of creating unified procedures, and the BPT
must establish those guidelines, ideally to be incorporated into the SOPs. The
chapter closes with recommendations for how Quest Diagnostics can sustain
consistency and continue to elevate performance across the company.
55
6.1 CLIA regulations regarding quality control
procedures As discussed in Chapter 2, Quest Diagnostics is a laboratory regulated under the
Clinical Laboratories Improvement Act, or CLIA. Laboratory practices must meet or
exceed the requirements set forth in CLIA, and understanding these regulations is a
critical foundation for understanding quality control practices at Quest Diagnostics.
The components of CLIA relevant to quality control are summarized here.
i. Quality control procedures must monitor accuracy and precision of
tests that may be subject to instrument drift, environmental changes,
or variance in instrument or operator performance over time.
ii. Procedures must follow or exceed the number and frequency specified
by the equipment manufacturer subject to the constraints listed below.
1. For quantitative assays (i.e., those that report a concentration
of target compound in the patient sample), quality control
samples must be run at two different concentrations every day
patient testing is performed.
2. For qualitative assays (i.e., those that report either a positive or
negative diagnosis), positive and negative quality control
samples must be run every day patient testing is performed.
iii. The laboratory must establish criteria for accepting quality control
results, and the criteria for quantitative tests must be based on
statistical parameters (e.g., mean and standard deviation) for each
batch.
iv. The laboratory must follow the manufacturer’s specifications for using
reagents and be responsible for the results.
v. The laboratory must document all quality control tests performed.
6.2 Quest Diagnostics quality control
requirements Quest Diagnostics fulfills CLIA requirements by establishing standard operating
procedures (SOPs) for quality control for each type of test performed on the target
platform, as summarized in this section. In addition to meeting CLIA requirements
56
and manufacturer specifications, Quest Diagnostics quality control requirements
intend to lead to accurate patient results in a high-volume, rapid-turnaround-time
environment.
The Quest Diagnostics SOPs for the target platform require:
i. For qualitative tests, negative and positive control samples must be
run at the beginning of every day of testing, and positive control
samples must be run intermittently throughout testing and after the
final patient test of the day, thus bracketing each batch of patient
samples between control samples.
ii. Similarly for quantitative tests, the low level (target compound is not
present) and the high level (target compound is present at a clinically
significant concentration) control samples must be run at the beginning
of every day of testing, and high level control samples must be run
intermittently throughout testing and after the final patient test of the
day, again bracketing patient samples between control samples.
iii. Passing negative quality controls must have a negative result, positive
quality controls must have a positive result, and quantitative results
must fall within plus or minus of the identified standard deviations of
the mean for the quality control material batch as tracked by Quest
Diagnostics.
iv. Patient results that are not bracketed by successful quality control
tests must be rerun, and operators must document corrective actions
taken; however, the laboratory director or designated authority may
override the rejection of runs and document the justification.
Quest Diagnostics does not establish policies regarding a preferred type of container
to be used for quality control material, a policy on whether quality control materials
may be reused across multiple batches, or a set of criteria for overriding the rejection
of runs; and CLIA quality control regulations do not require a laboratory to establish
rules in these areas.
Note that because of the mechanics of the testing platform, the instrument always
yields a numeric result representing the factor by which the measured luminesce
exceeds the detection level. Therefore results on this platform of “reactive” (positive)
and “nonreactive” (negative) are supported by a numeric measure. This creates the
57
possibility – if approved by the appropriate internal medical experts – of using
quantitative acceptance criteria for qualitative assays, despite the qualitative nature
of the assay.
Maintaining SOPs for each testing area (e.g., hematology, immunology, etc.) falls to
a interdisciplinary teams consisting of experts in medicine, regulations, and
laboratory operations called the Best Practices Team, or BPT.
6.3 Results from quality control survey As discussed in Section 5.1, the four laboratories observed in this study performed
testing consistent with the SOPs; however, because Quest Diagnostics has not
established a standard for all elements of operations, such as a preferred test tube for
quality control testing, laboratory practices differed in subtle but significant ways. A
survey was sent to all laboratories using the target platform to gauge the extent of
variation in quality control frequency, container size, acceptance criteria, reuse
policy, storage method, and approach to mechanical errors. The survey questions are
shown in Section 5.1.1.1.
The results of this survey are as follows:
Table 6.1: Frequency of quality control use
This table shows the fraction of Quest Diagnostics laboratories running the indicated
type of quality control material at the specified frequency. All Quest Diagnostics
laboratories either meet or exceed quality control requirements. The laboratories
exceeding quality control requirements are shown in bold italics.
Testing
frequency
Type of QC
material
Intermittently
throughout
patient testing
Every shift Once per day
Positive/high values 100%
Negative/low values 25% 25% 50%
Recall that the SOPs require positive / high values of quality control intermittently
throughout patient testing and negative / low values of quality control once per day.
The boldfaced values in Table 6.1 indicate the labs with practices consistent with
the SOP. 50% of labs test negative / low values of quality control several times more
often than is required.
58
Table 6.2: Quality control material acceptance criteria
This table shows the fraction of laboratories employing the indicated criteria for
determining whether to accept the quality control results.
Acceptance
criteria
% of
labs
Same as test 60%
Quantitative 40%
CLIA regulations set the sample acceptance criteria for quality control results: for
qualitative tests, the positive sample yields a positive result and the negative sample
yields a negative result; and for quantitative tests, the numeric results must fall
within an acceptable range of the established mean value. Standard industry practice
sets the tolerance window according to the Westgard rules, specifically, plus or
minus two standard deviations, as established by historical results, of the accepted
mean. As shown in Table 6.2, 60% of Quest Diagnostics laboratories set their
acceptance criteria (qualitative vs. quantitative) based on whether the test
performed generates qualitative or quantitative results. The remaining 40% use
quantitative criteria for all tests, regardless of the type of result reported, which
exceeds the minimum standards for tracking.
Table 6.3: Reuse policy and container size for quality control material
This table indicates the fractions of laboratories that prepare fresh containers of
quality control materials for every use (“No”), those that reuse a container of
quality control material and dispose of it when only the dead volume remains (“Yes
no repl”), and those that reuse a container of quality control material and replenish
it from a parent container of quality control material when the level gets low (“Yes
w repl”). The results are further broken down by the size of the container used by
those labs, with the dead volume associated with each container shown on the right.
Reuse Policy Dead volume
(uL) Container Size No
Yes no repl
Yes w repl
0.5mL 15% 100
12x75mm 45% 15% 300
13x75mm 5% 5% 300
16x100mm 5% 4500
16x85mm 5% 5% 450
59
Quest Diagnostics does not specify a preferred container for quality control material
during testing and laboratories have established a variety of procedures. The choice
of container greatly impacts the amount of “dead volume”, i.e., the quantity of
material that the instrument cannot access and must be discarded after testing.
Quality control materials are also tracked as part of the reagent waste initiative, and
reducing the amount of discarded quality control material would generate savings.
In addition, Quest Diagnostics does not establish a company policy on quality
control material reuse across batches, so each laboratory establishes its own policies.
The choice of container size combines with the laboratory reuse policy to drive large
variations in the amount of quality control material consumed by a lab. For
example, as shown in Table 6.3, the 15% of labs that reuse quality control
materials in a 12x75mm test tube and replenish the container from a master
container will discard about 300uL of quality control material once in several days,
whereas the 5% of labs that prepare a 16x85mm test tube of quality control material
for every batch discard at least 450uL several times per day. The volume of quality
control material varies depending on the test but is always less than 300uL. Thus
the laboratory that reuses and replenishes a quality control container will discard as
little as 1% of the quality control batch, compared with the laboratory that does not
reuse quality control material, which discards over 60% of the quality control
material batch.
The primary concern associated with reusing quality control material is the
possibility of contamination or evaporation leading to incorrect results.
Contamination could affect negative quality control samples, yielding incorrect
results. Evaporation could lead to elevated concentrations of the target compound in
the positive control sample, which could lead to results exceeding control limits for
tests tracked quantitatively. The best ways to prevent both contamination and
evaporation are refrigerating the quality control material and capping the container
between batches, but laboratories vary in their treatment of quality control
materials. Table 6.4 shows the fraction of labs that refrigerate and/or cap the
quality control containers between uses. The values do not sum to 100% because the
25% of labs that do not reuse quality control materials are excluded from this table.
60
Table 6.4: Reused quality control material storage policies
This table shows results for only the laboratories that indicated that they reused
quality control materials across multiple uses. The results indicate the fraction of
laboratories refrigerating and/or capping the reused quality control containers
between uses.
Cap
Refrigerate No Yes
No 5% 10%
Yes 5% 55%
Because mechanical errors occur on average once in every 23 samples (4.3% of the
time), occasionally a transient error occurs on a quality control sample. The SOPs
require that labs must successfully test positive quality control samples
intermittently throughout patient testing or else all patient samples from the
affected assay must be repeated. However, the laboratory director may override the
patient retest requirement based on professional judgment. Labs tended to interpret
this rule in one of four ways. When transient errors occurred on quality control
samples, labs would do one of these four things:
1. Address the mechanical error and rerun quality control samples as soon
as possible;
2. Rerun all patient samples;
3. Rerun 5-10 positive patient samples on a separate machine and treat
agreement between the first and second round of results as quality
control; or
4. Choose between options 2 and 3 depending on the type of mechanical
error.
Table 6.5 shows the frequency of these policies within Quest Diagnostics
laboratories. The labs that opted to rerun all patient samples consumed 70-100 times
the reagent in addressing the error as the labs that opted to rerun only the relevant
quality control sample.
61
Table 6.5: Approach to mechanical errors on quality control samples
This table shows the fraction of laboratories employing one of four common
approaches to addressing a situation in which a mechanical error occurs during a
quality control sample.
Approach
% of
labs
Repeat QC 40%
Rerun 5-10 samples 15%
Rerun all 25%
Either 5-10 or all 20%
6.4 Tension between quality requirements and
reagent cost Setting quality control policies involves evaluating a range of costs and risks,
including reagent costs and the likelihood of incorrect results under the policy.
Generally a more conservative approach adds greater certainty with respect to the
quality of the patient results, but when this added benefit is marginal, changing to a
less conservative approach is a responsible way to reduce reagent costs. Similarly,
any action that can increase result quality without adding significantly to reagent
costs appears prudent. However, the trade-offs with evenly matched risks and
benefits require a level of industry expertise beyond the scope of this thesis. Any
recommendations presented in this thesis require the review of medical and quality
experts prior to implementation. This section includes a discussion of the risks and
benefits of variations in laboratory quality control practices and either a
recommendation or a deferral to the Best Practice Team.
Frequency:
The appropriate frequency of the negative quality control sample is influenced by (1)
the balance among the cost of quality control testing and the consequences of a
quality control test not meeting acceptance criteria; (2) the ease of identifying
potential instrument issues independent of quality control testing, and (3) the
complexity of testing positive and negative quality control tests with different
frequencies. Overall, these considerations suggest significant reagent cost savings and
few additional operational costs associated with a reduced frequency of negative
quality control testing.
First, we evaluate the trade-offs associated with testing costs. In an extreme
simplification of the diagnostic testing platform, we can model the instrument as a
62
system that can generate outputs (quality control results) that may drift out of the
control limits (plus or minus two standard deviations of the mean). For the purposes
of this cost assessment, we assume that if the instrument goes out of control, it
requires operator intervention to return to control. If the instrument does go out of
control, all samples in the previous batch must be reevaluated. In this case, the
expected cost of quality control testing after every batch is as follows:
Ctotal = b * Cneg + b * pneg * Crerun
Ctotal – total cost of quality control in a day
b – number of batches in a day
Cneg – cost of running a single negative quality control test
pneg – probability negative quality control sample exceeds control limits
Crerun – cost of rerunning a batch.
If the laboratory chooses to run the negative quality control every other batch, the
equation changes in three ways. First, the number of quality control tests decreases
by a factor of two. Second, the probability that any quality control test will exceed
the control limits increases by a factor of two because if the instrument goes out of
control during either batch, the quality control result will exceed control limits.
Third, the cost of rerunning the batches will double because every time a quality
control result exceeds the limits, two batches must be reevaluated. Thus the
expected cost of quality control testing becomes the following:
Ctotal = (b / 2) * Cneg + (b / 2) * (pneg * 2) * (Crerun,* 2)
= (b / 2) * Cneg + b * 2 * pneg * Crerun
A generalized form of this for quality control testing performed after any number g
of batches in a grouping is thus as follows:
Ctotal = (b / g) * Cneg + b * g * pneg * Crerun
To minimize the total cost with respect to the number of batches in a grouping, we
can then simply take the derivative of the cost equation and solve for the g where
the derivative equals 0:
dCtotal / dg = - b * Cneg / g2 + b * pneg * Crerun = 0
b * pneg * Crerun = b * Cneg / g2
g2 = (b * Cneg) / (b * pneg * Crerun)
g = sqrt(Cneg / (pneg * Crerun))
Thus the number of batches to be run before quality control testing is large when
the probability of the instrument drifting out of the control limits is small relative to
63
the ratio of Cneg/Crerun. Between April and December in one example laboratory, the
negative quality control sample exceeded the control limits for 0.145% of all tests
(0.00145), compared to a Cneg/Crerun ratio of approximately 0.07, an order of
magnitude larger. In this case, the ideal grouping is constrained by the frequency of
testing required by CLIA.
Moreover, the laboratories use an additional technique to identify potential
deviations that is more sensitive than the negative quality control sample. Positive
results occur much less often than negative results, and Quest Diagnostics tracks the
relative occurrence rates for the tests performed on this platform. If the positive
result rate significantly exceeds historical averages, a technologist will review the
entire batch and may rerun it, regardless of successful quality control tests.
Statistically, these average rates are much more likely to identify this malfunction
than a single control sample.
Finally, some laboratory employees expressed concern about a quality control process
in which positive and negative quality control samples were evaluated with different
frequencies because of the added complexity for the operators. This is not of great
concern, given the skill level and general procedural competency of the operators:
they consist primarily of college-educated technologists who are required to interpret
test results with a high level of sophistication. Adding a checklist or creating
separate sample carriers for the first quality control samples of the day – both simple
changes – could reinforce the different processes.
A simple reagent cost optimization model indicates that Quest Diagnostics should
run as few negative quality control samples as possible, given the historical frequency
of negative quality control samples exceeding the control limits. In this model, the
risks associated with this change would likely be small, given the secondary method
of evaluating quality and the high operator skill level. As a result, Quest Diagnostics
laboratories should consider running negative control samples at the minimum
frequency of once per day in all laboratories. This process change must be reviewed
by medical, laboratory, and quality experts within Quest Diagnostics before
implementation.
Acceptance criteria:
The survey results indicate a fairly even split in approaches to quality control
acceptance criteria. Labs that prefer uniformly quantitative methods argue that an
unusually high or low result in a qualitative test may nevertheless indicate a problem
64
with the instrument or with the quality control material and should therefore be
investigated.
An alternative approach could harness the benefits of quantitative tracking without
the costs of over-conservative acceptance criteria: labs could record the quantitative
results but use the qualitative results as the acceptance criteria unless there are
additional reasons to mistrust the results. For example, if a high result for a positive
control occurred concurrently with an unusually large number of positive patient
tests, the affected assays could be rerun. Without any indication of impact to patient
results, the offending quality control sample could be rerun alongside a second
quality control sample from a new batch to confirm alignment. A large difference in
result would indicate a problem with initial quality control material rather than the
instrument. Agreement between the samples yielding a normal value may indicate
transient errors affecting the results, so retesting the patient samples from the
affected test would be appropriate. Agreement between the samples yielding a high
value may indicate a need for recalibration, and retesting the patient samples would
again be appropriate. This approach would maintain quality standards by requiring
laboratories to investigate abnormal quality control results but would give them the
flexibility to accept patient results on the basis of qualitative criteria when
investigation does not lead to any reason to suspect the quality of the results. This
flexibility would eliminate the cost and time pressures associated with unnecessarily
running several hours’ worth of patient tests.
Reuse policy:
Reusing quality control materials increases the risk of contamination or evaporation.
However, if these risks are minimized through laboratory practices such as capping
and refrigerating the test tube in between batches, a consistent policy of reusing
quality control materials would reduce reagent costs by reducing dead volume
discarded between tests.
In order to determine the feasibility of reuse, I recommend Quest Diagnostics
perform a side-by-side comparison of laboratories using the three different
approaches: no reuse, reuse without replenishment, and reuse with replenishment.
One example laboratory that followed a policy of no reuse at the time of this thesis
maintains records of the outcome of every quality control test, indicating, for
example, whether the quality control sample exceeded control limits, a mechanical
error interrupted quality control sample processing, or the instrument required
recalibration. Between April and December 2016, this laboratory positive quality
control samples exceeded control limits in 0.132% of tests. Comparable data should
65
be obtained from laboratories with the other two reuse policies. If the laboratory
with a no-reuse policy experiences a significantly lower rate of abnormal quality
control results than the other laboratories, then we could conclude that evaporation
or contamination from reuse affects the results of quality control samples. On the
other hand, if there are no significant differences in abnormal quality control results,
we could conclude that reusing quality control materials has negligible impact on
result quality and therefore would offer a low-risk approach to reducing reagent
waste. Identifying the superior approach would allow Quest Diagnostics to improve
laboratory operations, either through improved quality or through reagent cost
savings.
Storage approach:
As discussed in the previous section, the storage approach goes hand in hand with
the reuse policy. Refrigerating and capping are irrelevant if fresh quality control
material is prepared for every use; but with a reuse policy, they are easy ways to
minimize the possibility of contamination and evaporation. The added complexity of
maintaining caps and remembering to refrigerate is relatively small and could be
minimized by using standard test tubes for the quality control material so that test
tube caps are readily available or color-coding the caps to differentiate them.
Container size:
As with storage approach, the best container size depends on the reuse policy.
Without reuse, the best container is the 0.5mL cup used by 10% of labs because it
yields the smallest amount of dead volume – 100uL per sample. With reuse, the best
container is the smallest container that also has a cap, i.e., the 12x75mm test tube.
Making the quality control container uniform across Quest Diagnostics involves
essentially no cost other than the time associated with making the recommendation,
especially because 60% of labs already use the 12x75mm container, as shown in
Table 6.3. With this change, dead volume would consume a factor of 15 less than
the largest container currently in use requires.
Approach to mechanical errors on quality control samples:
Determining the best response to a transient mechanical error on a quality control
sample requires expertise beyond the scope of this thesis. This section will explore
the different approaches observed in Quest Diagnostics laboratories and highlight the
importance of unifying guidance from the Best Practice Team.
As shown in Table 6.5, the labs respond in four different ways, with responses split
fairly evenly:
66
1. Rerun quality control samples;
2. Rerun all patient samples;
3. Rerun 5-10 positive patient samples; or
4. Choose between options 2 and 3.
The first option represents the best choice because it requires only one additional
test. However, it is not always possible. For example, if the mechanical error
happens close to the end of the acceptable interval within which a quality control
test must be performed (see Table 6.1 which describes the recommended quality
control test interval), the fastest repeat test might not yield a result until an hour
after the previous quality control result or later because of the extended incubation
time required on the target platform. At this point, the options available include
numbers 2-4 above.
Option 2, rerunning all patient samples, represents the most conservative approach.
This strategy incurs large costs in terms of reagent use, operator time, and increased
turnaround time.
Option 3, rerunning 5-10 confirmatory patient samples, represents a more cost- and
time-sensitive strategy, representing an order of magnitude decrease in the reagent
required compared to Option 2. The 5-10 patient samples, usually with positive
results (i.e., target compound is present) to replicate the positive control, are
retested on a separate instrument. If the results match the original diagnoses, the
agreement could be interpreted as sufficient evidence that the precision of the
instrument is sufficient; and if the patient samples are bracketed by successful
quality control samples, then accuracy of the results can be confirmed as well. To
bolster the argument in favor of this approach, the operator could evaluate the type
of mechanical error based on the information provided in the condition code and
determine whether the error might have affected the outcome of other results. Recall
that in general, transient errors will cause the instrument to jettison a test and
report no result because of the uncertainty in the quality of the result. Therefore
most transient errors would be unlikely to generate inaccurate patient results and
would instead provide no result.
The question of whether repeating 5-10 patient samples is acceptable quality control
process ultimately falls to medical, laboratory, and quality experts within Quest
Diagnostics, who will evaluate the risks and costs associated with such a policy. In
general, creating greater consistency among these quality control practices can lead
to reduced reagent waste or improved reliability of patient testing.
67
6.5 Implementing quality control procedure
changes
6.5.1 Changes at an example laboratory
As discussed in the previous section, many of the variations observed in quality
control procedures involve the expertise of the BPT and therefore could not be
implemented during the timeframe of the internship. However, one facility plans to
modify the frequency of negative quality control samples, per the recommendations
above.
One facility evaluated the following considerations before implementing the change.
First, the reagent costs of running a negative quality control sample after every
batch were compared to the reagent costs of running the negative quality control
sample at the minimum required frequency. The potential reagent cost savings were
considered large enough to justify further exploration. Second, the frequency of
negative quality control results exceeding the specified control limits was evaluated.
The low frequency suggested that the additional negative quality control samples did
not provide meaningful information about result quality above and beyond the other
quality control checks performed for instrument results. Third, the impact on the
process of reporting results was evaluated and determined to be minimal. Finally,
the policy change was confirmed by the appropriate medical and laboratory experts
and implemented.
6.5.2 Savings due to changes
The example facility immediately saw a significant decrease in the fraction of reagent
consumed in quality control. The change went into effect mid-October, and Figure
6.1 shows the steep drop in the fraction of reagent used on quality control from 6-
7% in March-June down to 3-4% by November.
68
Figure 6.1: Rate of quality control use in example laboratory
The usage counter data showed that the rate of quality control testing decreased
from 6-7% during the four months preceding the project down to 3-4% by November.
Prior to the project the laboratory had been running negative quality control
material every 4 hours of patient testing. Based on the observations of the project,
the lab modified its policies in mid-October to run negative controls daily, consistent
with the standard operating procedures for the platform.
6.5.3 Continuing to align quality control practices
Variations in quality control practices arose on the target platform, and they are
likely to reoccur without measures to maintain alignment. Furthermore, variations
are likely to exist across the many other platforms used in Quest Diagnostics
laboratories, so any procedures to maintain consistency would benefit multiple areas
within the lab. In order to sustain consistent practices and continue to improve
quality control processes, I recommend the following:
1. Continue identifying non-uniformities in laboratory operating
procedures;
2. Provide more incentives for ongoing improvements; and
3. Update SOPs regularly with new best practices.
This thesis demonstrates the value of comparative analysis across Quest Diagnostics
laboratories. Quest Diagnostics should continue to perform this type of comparative
0%
1%
2%
3%
4%
5%
6%
7%
8%
9%
Dec Feb Apr Jun Aug Oct Dec
Instrument 1 Instrument 2 Instrument 3
69
evaluation to identify variations that arise in quality control practices on the target
platform and other platforms.
Improvements beyond the status quo come from the front lines. . Potential
innovations from the people who perform the task every day spark a healthy
reexamination of current processes. Quest Diagnostics should foster ongoing process
improvement. Quest Diagnostics should foster ongoing process improvement. For
example, the company could provide financial incentives to cost savings measures
that adhere to company quality standards and that the BPT subsequently approves.
As individuals generate process improvements, the BPT will need to update the SOP
on a regular basis. The BPT can include these updates as part of their regular
review process.
Together, these recommendations will help Quest Diagnostics align laboratory
practices around common best practices that continue to drive savings and
improvements in quality.
6.6 Chapter summary
The previous chapter identified quality control as one of the two primary sources of
reagent waste within Quest Diagnostics. Laboratory observation and surveys reveal
that laboratories across Quest Diagnostics take subtly different approaches to quality
control practices, leading to significant variations in the amount of quality control
material consumed at each site. The primary sources of variation are quality control
frequency, container size, acceptance criteria, reuse policy, storage method, and
approach to mechanical errors.
The optimal approach may be identified using existing data. For example, we
determined that the example laboratory could decrease the frequency of running
negative quality control samples without significantly increasing the cost of repeat
testing and while remaining in compliance with manufacturer guidance and CLIA
regulations. Similar intra- or inter-laboratory evaluations could determine the
effectiveness of quality control reuse, for example.
Achieving greater uniformity in operating procedures can help Quest Diagnostics
reduce reagent costs or improve reliability of patient testing. The company may be
able to reduce variations on an ongoing basis through the following
recommendations:
70
1. Continue identifying non-uniformities in laboratory operating
procedures;
2. Provide more incentives for ongoing improvements; and
3. Update SOPs regularly with new best practices.
The next chapter discusses the countermeasures for the second major source of
waste: mechanical errors.
71
Chapter 7 Countermeasures to eliminate mechanical error waste As discussed in Chapter 5, the two primary sources of reagent waste were quality
control and mechanical errors. Chapter 6 discussed strategies to reduce the former,
and this chapter will discuss strategies for the latter.
The current state analysis in Chapter 5 identified that operator skill levels varied,
that laboratories did not necessarily share best practices, and that mechanical errors
may be reduced through a maintenance program that goes above and beyond
manufacturer requirements. Thus the goals of this part of the project were to
standardize and elevate operator skill levels and to create maintenance standards
that exceeded manufacturer requirements. This chapter discusses the CLIA
regulatory framework within which these modifications can be made.
The author launched an Autonomous Maintenance (AM) pilot project within the
Marlborough laboratory in order to achieve the goals of this section. The AM team
completed Steps 0-2 of an AM program, creating several training documents related
to best practices and generating a set of trial maintenance procedures. The training
materials will be circulated within the company to achieve more consistent operator
skill levels. The trial maintenance procedures did not show a reduction in mechanical
errors because of the relatively short period of implementation.
Like the previous chapter, this chapter ultimately highlights the need for ongoing
incorporation of best practices into SOPs and training materials, and for incentives
related to process improvement.
7.1 Goals for AM pilot program The goal of this section of the project was to reduce reagent waste from mechanical
errors. Chapter 5 discussed a few key observations related to instrument
mechanical performance. Specifically, operator performance varied within and across
labs, the vast majority of mechanical errors occurred on four instrument subsystems,
72
and the most common of these errors could be addressed through maintenance above
the manufacturer’s recommended maintenance program. Together, these
observations suggest two strategies for tackling mechanical errors.
1. Increase the quantity or quality of information about best maintenance
practices provided to operators to ensure uniform performance of
maintenance; and
2. Develop additional cleaning and inspection procedures above and
beyond standard maintenance procedures to reduce the most frequent
mechanical errors.
As with the quality control standards, the countermeasures for mechanical errors
were developed with the intention of disseminating the results at a national scale to
maximize the impact of the proposed changes. Because these goals aligned with the
goals of Autonomous Maintenance, as discussed in Chapter 4, this project used the
structure of Autonomous Maintenance in order to address mechanical errors.
7.2 CLIA regulations related to operation,
maintenance, and equipment modifications CLIA regulations were less salient in tackling mechanical errors than in resolving
variations in quality control processes because the challenge here related more to the
mechanics of improving instrument performance than to varying interpretations of
regulatory requirements. Nevertheless, CLIA requirements form an important
foundation for our approach.
The relevant elements of CLIA requirements for equipment maintenance are as
follows:
- For equipment unmodified from the manufacturer’s original design, the
laboratory must perform maintenance as specified by the manufacturer
and with at least the frequency specified by the manufacturer. (493 CFR
1254.a)
- Quality control testing must be performed after any major preventive
maintenance or after replacement of any critical part that may influence
test performance. (493 CFR 1256.6)
- Laboratory personnel must document all maintenance performed. (493
CFR 1425.3)
73
- The laboratory must document evidence that operators have the skills
required to perform preventive maintenance and troubleshooting. (493
CFR 1423.4.ii.D)
The above regulations influence Quest Diagnostics maintenance procedures in the
following ways. First, the target platform, as installed in Quest Diagnostics
laboratories, remains in its original and unmodified form. As a result, any additional
maintenance activities must be performed with the guidance of the manufacturer.
Second, significant preventive maintenance activities trigger a round of quality
control testing. Ideally, then, preventive maintenance should occur during normal
breaks between rounds of quality control to minimize the amount of quality control
waste generated from maintenance activities. Third, supplementary documentation
must accompany any additional maintenance performed. Fourth, new training
materials must come with a mechanism for ensuring operator comprehension. The
countermeasures discussed in this chapter work within the guidelines set forth above.
7.3 AM team structure and activities To tackle waste from mechanical errors, this project launched an Autonomous
Maintenance (AM) program within the Marlborough laboratory focused on the
target platform. AM was an attractive approach for two reasons. First, the company
had recently hired an expert in Total Productive Maintenance (TPM) with the
intention of incorporating TPM into the company’s maintenance processes and
continuous improvement efforts. Therefore the expertise required for a successful
launch was readily at hand. Second, the strategies of the AM approach aligned with
the goals of this project, specifically, to increase operator skill levels and to develop
additional maintenance procedures.
This section discusses the structure of the team and activities within the AM
program.
7.3.1 Team structure
As discussed in Chapter 3, the recommended organization involves overlapping
small groups throughout the company: groups of 5-7 individuals focus on a particular
platform or area of the work space, and the leader of any group also participates in a
group organized at the next higher level of the organization. In other words, the
recommended structure for the AM team would consist of 5-7 operators focused on
the target platform, and the operator designated as the AM team leader would also
be part of a supervisors’ and managers’ AM team focused on a different platform.
74
However, resource constraints resulted in an AM team with one operator, a rotating
group of vendor representatives, and the author as the team leader, and no TPM
organization representative beyond the working group. Section 7.5 discusses the
impact of these limitations. The AM team met weekly for nine weeks to perform
activities that required access to the instrument. Outside of the weekly meetings,
team members performed additional activities related to team goals, as discussed in
the following section.
7.3.2 Autonomous Maintenance activities
The components of an Autonomous Maintenance program are well documented and
well characterized. The Quest AM team followed several of the procedures for Steps
0, 1, and 2 of Autonomous Maintenance with the goal of reducing mechanical error
frequency, as discussed in this section.
Step 0: Education
During Step 0, the goal of the AM team is to lay the foundation for subsequent
activities. The activities consist of establishing the structure of the AM team,
learning about the tools of AM, evaluating the current state of the instrument, and
developing a deeper understanding of how the instrument works, each of which will
be discussed in this section.
Establishing AM team structure: The first AM team meeting established individual
roles and responsibilities, guidelines and restrictions, and norms. For each type of
activity within a given AM step, the team members each took on responsibility of
maintaining that particular aspect, for example, collecting the supplies needed for
the group activity or performing safety audits before beginning work. The guidelines
and restrictions established the scope of activities. To minimize disruption to
production schedules and to maintain continuity across team meetings, we only
worked on a single instrument within the Marlborough lab with the assumption that
any findings on that instrument would be generalizable to other instruments and
other labs. We also set the boundaries of the type of activities that could be
performed based on Quest quality standards, e.g., that our activities could not
compromise the quality of patient results or impact compliance with CLIA
regulations, e.g., that the manufacturer had to approve any changes to maintenance
procedures. Finally, the team norms established expectations for how the team
would operate, such as participating actively, meeting deadlines, and fulfilling the
established responsibilities.
75
Learning tools of AM: This stage involved familiarizing the team with three tools
critical to AM: the principles of 5S, Quick Kaizens, and One-Point Lessons (OPLs).
The 5S principles are intended to empower operators to eliminate wasted energy,
time, and materials in their work. After receiving training from an expert in the
principles of 5S, the AM team applied those principles to create a standardized cart
of maintenance supplies to streamline the maintenance process.
The AM team leader trained the rest of the team on Quick Kaizens – improvements
to the efficiency, safety, or quality of current operations – and OPLs – concise
training documents that share (1) basic knowledge, (2) troubleshooting tips, or (3)
improvement ideas. Because these documents are designed to capture and share best
practices, the AM team identified them as a primary mechanism for sharing the
knowledge gained through AM activities and for disseminating best practices learned
at other sites. An example of an OPL is shown in Figure 7.1.
Figure 7.1: Example One-Point Lesson
Operators wash incubator rings as a part of routine maintenance. This lesson
demonstrates proper technique for drying the rings in order to avoid leaving behind
fibers, which can cause incubator jams.
76
Measuring baseline equipment condition: Before we could measure our progress, we
had to know where we were starting. We began by coordinating with the
manufacturer to perform a condition audit of the instrument. The AM team
established an audit checklist for all functional components of the instrument, and
an engineer from the manufacturer evaluated the instrument based on those criteria.
The audit revealed a small number of components that were not performing at
optimal levels and required separate inspection or replacement. The audit also
identified areas of the instrument where current maintenance practices as specified in
manufacturer instructions were not sufficient to prevent debris from accumulating to
levels that could potentially impact instrument performance.
Quantifying baseline instrument performance: The primary metric AM uses to
measure performance is Operational Equipment Effectiveness (OEE), as discussed in
Chapter 3. The metric is a product of Availability, Performance, and Quality,
where the capitalization is intended to denote the AM definitions for each of these
terms. Availability represents the amount of time the instrument operates relative to
the amount of time allocated for the instrument to operate. Performance represents
the number of samples processed during that time compared to the instrument’s theoretical throughput. Quality represents the fraction of successful jobs performed
during that time.
We were unable to identify reliable data sources for a reliable calculation of OEE
and therefore relied on other sources of information in order to measure progress.
Specifically, no data sources existed that could easily calculate Availability; and
Performance and Quality metrics were not available at the daily timescale most
useful for OEE. The AM team tested tracking Availability by asking operators to
record the amount of downtime for a target instrument, but this did not prove
successful: because of their large workload, operators struggled to record consistent,
accurate data about the duration of instrument downtime and its causes. Daily data
about Performance and Quality was only available through a labor-intensive process
of collection and processing, which we determined to be unsustainable for the AM
team and for the operators.
Therefore instead of looking at the holistic measure of OEE, we used two of the data
sources described in Chapter 5 to measure instrument performance. First, the
usage counter data substituted for a Quality metric at a weekly scale, showing the
fraction of tests used for patient results. Second, the condition code data replaced a
Performance metric by showing the number of breakdowns over the course of a
week. While the condition code data did not provide information about the amount
77
of downtime due to mechanical errors, the more important metric for the purposes of
this thesis was the impact on reagent waste, which could be inferred from the data.
Learning instrument mechanics and safety: The final components of Step 0, learning
the instrument mechanics and establishing safety guidelines, laid the foundation for
subsequent team activities. With the guidance of vendor engineers, the AM team
first established a safety checklist for the instrument and the surrounding area to
guarantee that we could perform work on the instrument with no risks to our health
and safety and no risks of damaging the equipment. Parts of the instrument with
inherent risks were identified as a potential target for Quick Kaizens or, if the AM
team could not eliminate the risk, as a point of design feedback for the
manufacturer. The AM team reviewed the safety checklist before performing any
activities.
Then the AM team discussed the mechanics of every part of the instrument.
Following standard AM practice, the team forced themselves to evaluate every
component by creating detailed sketches of the instrument, which promoted
attention to detail and a methodical review of components.
Step 1: Clean to inspect
During Step 1, the goal of the AM team is to expose hidden defects within the
instrument in anticipation of updating cleaning and inspection standards in Step 2.
Activities in Step 1 include cleaning and inspecting the instrument over time;
identifying and, where possible, eliminating areas of the instrument that are difficult
for the operator to access; and tracking any Sources of Contamination (SOCs). Hard
to access areas can be eliminated by making the areas more accessible or identifying
procedure changes to prevent contamination from reaching the area, thereby
precluding the need for access. Note that in this context, the capitalization of SOCs
is intended to denote the AM-specific definition of contamination, which is simply
any substance found where it is not intended, and not to connote any danger for
contaminating samples.
Cleaning to inspect: Just as drawing helped the AM team more deeply investigate
different instrument components, manually cleaning every instrument component
encouraged the AM team to thoroughly review the current instrument condition.
The AM team performed this activity for three weeks. The first cleaning established
a spotless baseline condition so that any Contamination that accumulated over the
course of the week could be easily identified and traced to its source. The approach
follows the same logic as a car owner who wants to keep her driveway clean to detect
78
oil leaks: if the driveway is dirty and covered with oil splatters, she will struggle to
notice any new drops, but if the driveway starts out clean, she can easily detect any
new drops and address the oil leak immediately.
Identifying hard-to-access areas: In anticipation of a Step 2 goal of reducing the total
time required for maintenance, Step 1 involves identifying areas of the instrument
that are difficult, uncomfortable, or unpleasant to access. Examples include parts of
the instrument that are difficult to clean, such as a shelf where disposed consumables
accumulate that is located at knee height and is the length of an arm, so, without
modifications to the instrument or tools used, cleaning the shelf requires the operator
to lean over or kneel down and insert an arm into the space; or information stored in
the instrument that is not readily accessible through the user interface. This activity
involved tracking these hard-to-access areas. When doing so did not involve an
unacceptable or infeasible modification of the instrument, the AM team developed
countermeasures to improve accessibility and documented the actions in the form of
Quick Kaizens or OPLs. Otherwise, the observations were submitted to the
manufacturer as potential targets for design changes.
Tracking Sources of Contamination: The process of cleaning to inspect the
instrument in this stage then allowed the AM team to track Sources of
Contamination. During the initial cleaning, the AM team recorded all debris
observed before cleaning it. During subsequent cleaning, the AM team noted whether
the debris had recurred. If not, then the debris was considered a one-time occurrence
that did not require additional maintenance measures to address. An example of this
type of debris is dust on horizontal, non-functional surfaces in the instrument, which
had accumulated since installation but would not have any impact on instrument
performance. In this instance, because a small amount of dust had accumulated over
a relatively long time, no countermeasures were necessary. If the debris appeared on
subsequent AM team sessions, however, a root cause analysis was required.
The root cause analysis allowed the AM team to categorize the debris as either
normal or abnormal. Consider, for example, debris that accumulates under a moving
component on the instrument, and the engineering representative from the
manufacturer identifies that the debris comes from friction between moving parts. If
the parts are designed to create friction, then this debris would be considered a
normal SOC, and an adequate countermeasure would involve monitoring the debris
and cleaning it regularly. If, on the other hand, the parts were not designed to wear,
the debris would indicate an abnormal SOC. Abnormal SOCs provide evidence of a
hidden defect, which requires immediate resolution. In this instance, the hidden
79
defect would be a problem with component alignment that would trigger remedial
action from a service engineer provided by the manufacturer. Following service, the
AM team would recommend that maintenance procedures also include checking for
this debris and triggering service if debris is observed, thereby establishing a process
for rapidly identifying the defect if it returns.
For every SOC observed, this process identifies:
1. A description of the debris and its location within the instrument;
2. A description of the root cause of the problem; and
3. A solution for how to address it going forward.
Step 2: Tentative Cleaning and Inspection Standards
During this step, the AM team synthesized the understanding of the instrument and
the observations of SOCs to develop cleaning standards that go above and beyond
the standard manufacturer recommendations. Traditionally this stage also includes a
goal of decreasing the total operator time spent on maintenance to ¡2% of the total
labor time. While the AM team could reduce the total amount of debris
accumulation through eliminating hidden defects and addressing hard-to-access
areas, we could not eliminate cleaning steps that are part of the vendor’s validated
process for performing instrument maintenance.
Over the four weeks available for this step, the AM team developed supplemental
cleaning standards. We prioritized improving maintenance for instrument subsystems
with the highest number of mechanical errors, as determined using the condition
code data. To address the mechanical errors, we combined an assessment of the most
frequent condition codes for that subsystem with observations of debris accumulation
inside the instrument to identify the specific component or process that the cleaning
should target. After developing the cleaning standards and receiving manufacturer
approval to proceed with them, we trained all operators responsible for maintenance
on the new procedures and, in compliance with CLIA regulations, added
documentation of the new maintenance steps to the standard maintenance tracking
system.
The new maintenance standards applied only to the instrument that was the focus of
the AM team’s activities to allow direct comparison of performance between the
“test” instrument, where the new standards were applied, and the “control” instrument, where only the original maintenance procedures were used.
80
7.4 Outcomes
7.4.1 Impact on instrument performance at
Marlborough
The goal of Step 2 was to develop cleaning and inspection standards that improved
instrument performance. Unfortunately, both usage counter data and condition code
data indicate that the AM team did not accomplish that goal, as shown in Figure
7.2. In retrospect, this outcome is unsurprising given the short timeframe available
to design and implement potential changes.
Figure 7.2: Rate of mechanical errors in Marlborough
The usage counter data did not indicate that the frequency of mechanical errors on
the test platform, Instrument 3, decreased significantly after the beginning of the
AM program: note the results for October and November are comparable to those of
the preceding months, except February, which experienced an abnormally high rate
of errors for reasons beyond the scope of this thesis.
Because the maintenance countermeasures were not having a significant positive
impact on instrument performance during the final week of the pilot program, a
longer-term approach was established in order to continue the process of improving
maintenance. While the AM team meetings could not continue absent the AM team
leader, a smaller working group was arranged: the operator representative and the
operator from the AM team allocate an hour each week to work on ongoing process
0% 1% 2% 3% 4% 5% 6% 7% 8% 9%
10%
Jan Apr Jun Aug Oct Dec
Instrument 1 Instrument 2 Instrument 3
81
improvement activities. They use this time to review the condition code data from
the prior week, discuss with vendor representatives how to address the most common
mechanical errors, and train other operators on the applicable modifications to the
maintenance procedures. The results of these efforts cannot be quantified at this
point because the improvement activities are ongoing.
7.4.2 Description of training materials and how
training materials were disseminated
While the tentative cleaning and inspection standards were not able to demonstrate
improvement in timeframe of the thesis, the AM activities had the positive effect of
increasing operator knowledge of the instrument.
Over the course of the AM sessions, the team collectively prepared 29 OPLs, which
were approved for general circulation by the vendor representatives. The OPLs will
be circulated through a national competency website. Individual laboratory managers
will be responsible for identifying personnel within their laboratory who are certified
to run the target platform and adding those operators to a central database. Then a
representative from the BPT can easily distribute OPLs to all operators trained on
the target platform over several weeks. The OPLs will be associated with
comprehension questions, which will verify that the operators absorbed the
information captured in the OPLs.
This process will allow widespread dissemination of the best practices captured
through the AM team meetings. Moreover, it creates a pathway through which sites
can share best practices related to instrument operation or maintenance. As
discussed in Chapter 5, one of the main preliminary observations from this project
was that labs did not regularly share best practices. By establishing the national
competency testing website as the means of communication and OPLs as the format,
this process will demonstrate to labs how these best practices can be shared and
encourage them to follow suit.
7.5 Discussion of implementation approach
7.5.1 Challenge of improving processes in a lab with
high productivity goals
A primary constraint for the AM team was the amount of time the operator on the
team could dedicate to the group activities. Because the laboratory faced increasing
pressure to maintain productivity and meet turnaround time requirements the
operator was usually responsible for running an instrument or set of instruments
82
concurrently with AM team meetings. Moreover, the operator did not have dedicated
time outside of the team meetings to explore improvement activities, such as
developing OPLs or implementing Quick Kaizens.
There are several potential strategies for addressing this type of constraint.
1. Ask the operator to “make time” during normal working hours;
2. Provide overtime compensation for process improvement activities;
or
3. Include process improvement as part of job expectations.
Consider the first option. Every job has stretches of time when the worker can
operate at a slower pace. One option for addressing the time limitations would
involve asking the operator to identify these slower moments as opportunities to
work on separate process improvement activities. However, given the high
expectations for work quality and turnaround time, adding to additional
responsibilities is likely to lead to operator burnout.
The second option decreases the burden on the operator during working hours by
providing overtime compensation for process improvements completed outside of
normal working hours. While this eliminates the stress associated with the multi-
tasking the first approach requires, it nevertheless adds strain on the employee. An
employee with significant demands outside of work, such as a child or an ailing
family member, would have significantly less latitude to contribute to process
improvements than ideal. The Marlborough lab followed this approach during the
AM pilot program, and the operator found time for the extracurricular work only on
occasion, despite the compensation incentive.
The final approach would involve including process improvement activities as part of
explicit job requirements for operators and allocating time during standard working
hours for these activities. This approach is likely to decrease the apparent
productivity in the short term as employees spend less of the total week performing
their primary front-line tasks. However, the time spent improving processes,
increasing safety, or strengthening training materials represents an investment that,
when implemented correctly, are likely to pay dividends in more efficient operations,
lower productive hours lost to workplace injuries, or lower rates of mechanical
failures.
Managerial support can be achieved by demonstrating the potential positive impact
of “quick wins”, i.e., improvement activities that provide a rapid return on the
83
investment of time, under either of the first two regimes as a way of justifying
additional investment. While this approach may require greater investment on the
part of company management, the returns are likely to outweigh what they would be
under the first two scenarios without the potential employee burnout likely under
those approaches.
7.5.2 Strategy for sustaining improvements
Just as with quality control processes, the gains from this investigation of mechanical
errors are best secured through ongoing comparative analysis across sites and
through incentives to contribute process improvement ideas, here in the form of
OPLs.
Comparative analysis: As observed in the sites visited, individual sites will continue
to develop innovative solutions to tackle process challenges. Continued comparative
analysis across sites will help to unite laboratory operations around a single best
practice.
Process improvement incentives: Ultimately, the most efficient process would involve
individual laboratories broadly communicating when they develop a process
improvement. For example, at the time of implementation when a laboratory group
communicates the process improvement internally, it could circulate the
improvement to the comparable groups in other laboratories within Quest
Diagnostics. In this paradigm, the labs self-identify when they have made
improvements and submit their improvements in the form of OPLs to some
approving body, such as the BPT, after receiving approval from the manufacturer.
Quest could begin the transition to this type of process immediately by creating
financial incentives for individuals or laboratories for developing OPLs that get
incorporated into the centralized database. This type of incentive would encourage
labs across the company to invest more in the long-term process improvements that
pay dividends across the company, rather than focusing exclusively on cutting
headcount and reducing waste.
7.6 Chapter summary As discussed in Chapter 5, mechanical errors account for approximately 4.3% of all
reagent use in Quest Diagnostics laboratories, all of which is considered waste for
purposes of this thesis. Many of those mechanical errors arise because of debris
accumulation at various subsystems within the instrument and therefore may be
addressable through more frequent or more intensive maintenance beyond what is
84
required by the manufacturer. The AM team attempted to develop such cleaning
standards that go above and beyond manufacturer requirements. While the AM
team was not able to demonstrate significant reductions in mechanical errors over
the course of the thesis, the approach established a good model for continuing efforts
to improve maintenance practices.
The secondary goal of the AM team activities was to standardize operations around
best practices. This motivation arises from the observation discussed in Chapter 5
that individual labs develop innovative solutions to operational challenges but do not
have a clear channel for disseminating these innovations to other laboratories.
Through the OPLs, the AM team documented best practices identified by individual
labs in conjunction with the manufacturer. By sharing the OPLs on a national
competency platform, Quest Diagnostics can more effectively disseminate best
practices of operators across the company and unify operations around these
innovative best practices.
The greatest challenge the AM team encountered was the constraints associated with
operator availability during the thesis. This challenge is the result of a different
component of the Invigorate initiative that emphasizes increasing employee
productivity instead of reducing reagent waste. In order to balance the potentially
conflicting goals of improving processes and increasing productivity, Quest
Diagnostics could include process improvement activities as part of employee job
expectations, thereby creating a structure for managerial support of these activities;
and they should offer incentives for improvement ideas to further inspire innovation.
85
Chapter 8 Conclusions This thesis aims to reduce waste on a high-volume diagnostic testing platform at
Quest Diagnostics. In this concluding chapter, we review the main findings of the
project, the countermeasures implemented, the recommended follow-up actions for
the company, and the most fruitful areas of future research.
8.1 Summary of main findings This section will briefly review the primary findings discussed in the previous
chapters.
1. Quality control and mechanical errors were largest sources of waste.
Data provided by the manufacturer indicated that quality control
samples and incomplete tests accounted for 5.2% and 4.3% of all tests,
respectively, from March through May 2016 across all Quest
Diagnostics laboratories. These represented the main sources of reagent
waste for the target platform.
2. Individual laboratories implemented quality control practices that
exceeded Quest Diagnostics operating procedures and regulatory
requirements
Survey results from all laboratories using the target platform indicated
that laboratories varied in their implementation of standard operating
procedures (SOPs). The variations included differences in quality
control frequency, container size, acceptance criteria, reuse policy, and
approach to mechanical errors.
The different approaches fell along a cost-conservatism spectrum that
universally met SOP requirements but occasionally exceeded those
requirements without substantial benefit to test quality. For example,
several laboratories ran negative/non-reactive quality control samples
more often than required, which, other laboratories argued, provided
86
little meaningful information about the accuracy of the instrument
results.
3. Mechanical errors were a major source of irritation for operators
The observed rate of mechanical errors may understate the impact of
these errors on the operator experience of the instrument. Anecdotally,
operators reported that the instrument experienced a relatively high
rate of errors, including occasional shutdown errors that generally
compromised all of the tests currently in progress.
4. The most common mechanical errors may be addressed through
improved maintenance.
Many of the most common mechanical errors arose because of
accumulation of debris that interrupted normal operation of the
instrument component. Thus increasing frequency or intensity of
maintenance beyond the manufacturer’s recommendation may reduce
the frequency of these mechanical errors. This observation suggested
that a program of Autonomous Maintenance could benefit Quest
Diagnostics by creating an independent team dedicated to improving
maintenance practices.
5. Laboratories do not share best practices related to maintenance or
process improvements.
Subtle differences between individual operators’ practices and between
overall lab practices related to maintenance and operation suggested a
lack of knowledge sharing both within and among laboratories. This
observation bolstered the argument for an Autonomous Maintenance
program, which involves creating training documents to capture and
disseminate best practices.
8.2 Recommendations for Quest Diagnostics This project developed countermeasures to address the opportunities that the above
observations revealed. This section discusses three sets of recommendations based on
the project analysis: (1) the actions Quest Diagnostics must take in order to lock in
the opportunities identified through this project, (2) the organizational changes
Quest Diagnostic should make to build from the learnings in this project, and (3) the
technical changes to data collection that will facilitate process improvement going
forward.
87
8.2.1 Specific process/decision-based
recommendations
The assessment of sources of reagent waste lead to two concrete recommendations to
address the immediate causes:
1. Add specificity to quality control procedures
2. Share best operation and maintenance practices across the company
The first recommendation will create greater consistency among laboratories in their
interpretation of quality control procedures. Depending on the best practice
ultimately determined by internal medical, laboratory, and quality experts,
standardizing these procedure beyond the level currently specified in SOPs may also
reduce reagent costs by avoiding over-conservative quality control practices that
provide little meaningful insight into result quality. The second recommendation will
reduce mechanical errors by elevating instrument operation to a consistently high
level.
Add specificity to quality control procedures
The most reliable way to unify laboratory quality control practices for the target
platform involves the Best Practice Team incorporating the recommended
approaches into SOPs for the platform.
Share best operation and maintenance practices across the company
This project began documenting and circulating best practices related to operation
and maintenance of the target instrument. In order to reap the full benefit of these
materials, Quest Diagnostics should circulate the training documents to everyone
involved in operating the instruments. The Best Practice Team is a natural
authority to oversee this training.
The One-Point Lessons capture many best practices, but operators will continue to
develop new ideas for improving processes on the target platform as well as other
platforms. The BPTs for different testing areas should encourage operators to
continue developing OPLs to be included in operator training requirements.
8.2.2 Organizational recommendations
This project highlights the opportunities related to aligning procedures and the
challenges of implementing change on one testing platform. To capture those
88
opportunities and overcome those challenges, Quest Diagnostics should make four
general organizational changes:
1. Continue performing comparative analysis
2. Establish process improvement objectives on par with clinical goals
3. Create incentives for problem-solving
4. Regularly update SOPs with best practices
Continue performing comparative analysis
Through four laboratory visits, this project revealed variations between laboratories
that could lead to substantial cost and reagent savings. Comparable variations
probably exist on other testing platforms, and Quest Diagnostics should continue the
type of comparative analysis performed in this thesis to identify and minimize these
variations.
Establish process improvement objectives on par with clinical goals
A primary challenge for the Autonomous Maintenance team activities arose because
of the higher priority placed on the clinical goals of turnaround times for patient
results and employee productivity despite top management’s support for process
improvement activities. A solution involves holding laboratory managers accountable
to process improvement goals in addition to their traditional goals. This would
encourage middle management to create more space for improvement activities. It
would also provide middle management with leverage against increasing productivity
requirements when those requirements compromise the process improvement goals.
Create incentives for problem solving
As observed in this project, operators have many ideas about process improvements
that can improve working conditions, increase productivity, or cut costs, all of which
create value for Quest Diagnostics. However, these local improvements do not
consistently spread to all operators within a laboratory or to all laboratories.
To encourage greater dissemination of these improvement ideas, Quest Diagnostics
should establish employee incentives for approved ideas. The process might proceed
as follows: the employee drafts a One-Point Lesson based on the proposed
improvement; the BPT reviews the OPL in the context of existing OPLs; if the BPT
approves the OPL, the lesson is incorporated to operator competency training, and
the creator of the OPL receives a cash bonus. Such a process would harness
employee creativity and benefit the company at a national scale at relatively low
cost.
89
Regularly update SOPs with best practices
The SOPs enshrine process requirements for all laboratories. In order to guarantee
that all laboratories follow the current best practices, the BPT for clinical testing
areas should incorporate newly identified best practices into SOPs as part of their
regular SOP review procedure.
8.2.3 Data collection recommendations
The improvement initiatives discussed in this analysis, especially those related to
reducing the rate of mechanical errors, rely on consistent, accurate data. Increasing
data precision and ease of access will help inform ongoing process improvement
activities and facilitate better decisions about how to make improvements. In fact,
the current lack of timely, relevant data will quickly become a roadblock as
operators try to improve maintenance processes.
To remove this barrier, we recommend that Quest Diagnostics prioritize generating
easy access to the following types of data in collaboration with the instrument
manufacturer:
1. Daily OEE data; and
2. Definitive mapping between condition codes and lost reagent.
Daily OEE data
An Autonomous Maintenance program cannot reach its full potential without daily
Operational Equipment Effectiveness data, which this project lacked. Each
component – Availability, Performance, and Quality – proved impossible or highly
labor-intensive to access on a daily basis. Given the time constraints already placed
on operators, any data collection step requiring more than a few minutes is
unsustainable.
Quest Diagnostics should therefore prioritize generating easy, daily OEE data. This
task will require collaboration with the instrument manufacturer, who has already
built a relationship with several laboratory personnel over the course of the AM pilot
project. While the manufacturer’s current data sources do not accommodate an OEE
calculation, there were several types of data that approached the correct information.
For example, the usage counter information was available on a weekly basis, but not
on a daily basis because of how frequently the instrument, which stored test-by-test
usage information, updated the manufacturer’s servers with new data. In this case, a
small software change could easily lead to daily Quality data.
90
Definitive mapping between condition codes and lost reagent
One key challenge of this project related to the difficulty of identifying which
condition codes that arose on the instrument led to incomplete tests. This project
relied in part on code-by-code input from vendor engineers and in part on
generalized observations about what made a condition code more likely to lead to an
incomplete result. Neither approach provided adequate consistency, ease, or
specificity for consistent process improvement of this type.
Unfortunately, the onus of this change lies again with the manufacturer’s software
team. The instrument software must be modified to correlate an incomplete result,
which is recorded in the usage counter data, with a mechanical cause.
8.3 Areas of further investigation In addition to the analyses of quality control procedures discussed in Section 8.2.1,
the above analysis suggests two follow-up areas of investigation:
1. Maintenance prevention through design improvements
2. Repeat comparative analysis on other instruments
Maintenance prevention through design improvements:
The observation that 89.7% of impactful mechanical errors arose from three
subsystems suggests that those subsystems are relatively prone to failure. While the
AM activities discussed in this project aimed to reduce the rate of mechanical errors
through additional maintenance procedures, a more efficient approach would involve
designing more robust components.
Comparative analysis for additional platforms:
The analysis of this instrument yielded many insights into potential improvements.
This type of analysis could be repeated across other platforms to generate
comparable reagent savings.
Together, these recommendations will allow Quest Diagnostics to address the
immediate causes of reagent waste, perform analyses to further improve
instrument performance and effectiveness of maintenance, and foster a culture of
continuous improvement.
91
Bibliography
[1] Quest Diagnostics, “2015 Annual Report on Form 10-K.” 2016.
[2] B. Wang, “Medical Equipment Maintenance: Management and Oversight,”
Synthesis Lectures on Biomedical Engineering, vol. 7, no. 2, pp. 1–85, Oct.
2012.
[3] Abhishek Jain, Rajbir Bhatti, and Harwinder Singh, “Total productive
maintenance (TPM) implementation practice: A literature review and
directions,” Lean Six Sigma Journal, vol. 5, no. 3, pp. 293–323, Jul. 2014.
[4] S. Nakajima, Introduction to TPM: Total Productive Maintenance.
Cambridge, MA: Productivity, 1988.
[5] Masaji Tajiri and Fumio Gotoh, Autonomous maintenance in seven steps:
implementing TPM on the shop floor. Portland, OR: Productivity, 1999.
[6] K. Shirose, TPM for Workshop Leaders. Cambridge, MA: Productivity, 1992.
[7] R. Attri, S. Grover, and N. Dev, “A graph theoretic approach to evaluate the
intensity of barriers in the implementation of total productive maintenance
(TPM),” International Journal of Production Research, vol. 52, no. 10, pp.
3032–3051, May 2014.
[8] H. M. Lazim, N. Ahmad, K. Hamid, and T. Ramayah, “Total employees
participation in maintenance activity: a case study of autonomous
maintenance approach,” Malaysia Labour Review, vol. 3, no. 2, pp. 47–62,
2009.