QC IN

BIOCHEMISTRY

Dr Reema Bahri, MDConsultant - QASRL Diagnostics Pvt Ltd, Mumbai

18th Dec’ 2011

QUALITY ASSURANCE VS. QUALITY CONTROL

Quality Assurance

Much more: An overallmanagement plan to guarantee theintegrity of data(The “system”)

Quality Control

A series of analytical measurements to ensure that the results generated by the test system are correct

Quality Control - QC refers to the measures that must be included during each assay run to verify that the test is working properly.

Quality Assurance - is concerned with much more: that the right test is carried out on the right specimen, and that the right result and right interpretation is delivered to the right person at the right time.

WHY DO LABORATORY ERRORS OCCUR?

QualityControl &

Assessment

PoorWorkload

Management

Understaffed

Non-validatedTests

InadequateAttentionTo Detail

Time Pressures

Poor Results Verification

Poor Sample Control

PoorQuality

Management

DEFINITION - QC IN THE MEDICAL LABORATORY

QC-Statistical process - to monitor & evaluate analytical process which produces

patient results to ensure that medical decisions can be taken with confidence

Ensures continuous monitoring of performance of test analytes

QC result - may be quantitative, qualitative (positive or negative) or semi-

quantitative (limited to a few different values)

A total QC system must control both trueness and precision

IQC and EQA (External Quality Assessment) are complementary in ensuring the

reliability of test results

? TRUE VALUE - UNKNOWN

True value - this is an ideal concept which cannot be achieved.

Accepted true value - the value approximating the true value, the difference between the two values is negligible or within acceptable limits

Error - the discrepancy between the result of measurand and the true (or accepted true value).

ACCURACY VS. PRECISION

AccuracyHow well a measurement agrees with an accepted value

PrecisionHow well a series of measurements agree with each other

PRECISE BUT INACCURATEIMPRECISE & INACCURATE

PRECISE & ACCURATE

Bias and imprecision are most important at the medical decision levels!

For example, ß-hCG clinical decision levels at low concentrations (corresponding to early pregnancy in the female and early testicular cancer in the male) or at moderate concentrations (to diagnose the progression of pregnancy)

HOW TO CARRY OUT ANALYSIS OF DATA?

Need tools for data management and analysis Basic statistics skills Manual methods

• Arithmetic Graph paper• Calculator

Computer helpful• MS Excel Software• Spreadsheet

QC Data Management Software programmes• Biorad Unity Realtime, Biorad Desktop • Randox 247• Medlab QC• Automated Analyzer software

Important skills for laboratory personnel

ESTABLISHING CONTROL RANGES

Select appropriate controls & use product insert ranges only as guidelines

Manufacturer Ranges are based on reagent lots and materials available at the time of value assignment. During the life of the control lot, manufacturers may reformulate tests or begin using a new source of raw materials for kit/reagent production.

Published ranges cannot account for variables such as instrumentation software updates or performance differences over time

Calculate Lab defined Mean & SD:

Provisional Mean - Collect database with min 20 data points from separate analytical runs Purpose - covers day to day sources of variability in the measurement procedure to be

reasonably represented in the mean value due to calibration frequency, change of reagent or reagent lot, operator technique, temperature/humidity of testing location, daily/weekly maintenance, etc.

If the desired 20 data points from 20 separate analytical runs are not available, provisional values may have to be established from data collected over fewer than 20 days

Four control measurements per day for five different days

A certain amount of variability will naturally occur when a control is tested repeatedly

The goal is to differentiate between variability due to chance than due to error

Determine the degree of variability in the data to establish acceptable range Calculate mean, standard deviation, coefficient of variation; determine target rangesPlot results to develop Levey-Jennings charts- allows you to visually review data

points plotted against a ±3SD range

The initial assessment of imprecision may not include measurement variability due to influence of factors that occur over a longer time period such as recalibration, reagent and calibrator lot changes, instrument maintenance, and environment variables

Therefore, this provisional mean and range can be followed till 90 days. Then recalculate lab mean and range over 90 days period to include all possible variations and apply till the end of lot.

New lot of control material is analyzed for each analyte of interest in parallel with the lot of control material in current use

Alternate QC - Tests for which calibration/control material is not available, following alternate quality control measures can be applied:

Retesting of any randomly chosen retained samples normal or abnormal

Retained proficiency testing material/reference material

Sample from healthy volunteer or staff known to be free from any disease

Replicate test of sample by different method, different machine and different person, wherever applicable

CALCULATION OF MEAN

Data set (30.0, 32.0, 31.5, 33.5, 32.0, 33.0, 29.0,29.5, 31.0, 32.5, 34.5, 33.5, 31.5, 30.5, 30.0, 34.0,32.0, 32.0, 35.0, 32.5.) mg/dL

The sum of the values (X1 + X2 + X3 … X20) divided by the number (n) of observations

The mean of these 20 observations is (639.5 20) = 32.0 mg/dL

MEASURES OF DISPERSION OR VARIABILITY

There are several terms that describe the dispersion or variability of the data around the mean:

Range Variance Standard Deviation Coefficient of Variation

RANGE

Range is the difference or spread between the highest and lowest observations

It is the simplest measure of dispersion

It makes no assumption about the central tendency of the data

NORMAL DISTRIBUTION

All values are symmetrically distributed around the mean

Characteristic “bell-shaped” curve

Assumed for all quality control statistics

STANDARD DEVIATION AND PROBABILITY

In general, laboratories use the +/- 2 SD criteria for the limits of the acceptable range for a test

When the QC measurement falls within that range, there is 95.5% confidence that the measurement is correct

Only 4.5% of the time will a value fall outside of that range due to chance; more likely it will be due to error

Take CAs

CALCULATION OF STANDARD DEVIATION SD - The standard deviation measures a test's precision or how close

individual measurements are to each other.

The standard deviation (SD) is the square root of variance or average squared deviation from the mean

SD is commonly used due to the same units as the mean and the original observations

SD is the principle calculation used to measure dispersion of results around a mean

A high standard deviation can be attributed to:

Inherent variability in the test, which represents expected error

Analytical system malfunction, which represents unexpected error that the laboratory must investigate and correct

COEFFICIENT OF VARIATION

The Coefficient of Variation (CV) is the standard Deviation (SD) expressed as a percentage of the mean

-Also known as Relative Standard deviation (RSD)

CV % = SD x 100 Mean

PRECISION: CV% VS. SD The CV is more accurate comparator than SD because latter

typically increases as the conc. of analyte increases A laboratorian can be easily misled if he is comparing precision

for two different methods (e.g. by instrument, method, reagent, etc.) by using standard deviation

Have a look: SD of hexokinase and glucose oxidase is 4.8 and 4.0 respectively.

Based on just SD, one might conclude that the glucose oxidase method is more precise than the hexokinase method

However, CV % for both methods is 4% which shows that the methods are equally precise. Assume the mean for the hexokinase method is 120 and the glucose oxidase mean is 100.

LJ WITH MANUFACTURER & LAB CALCULATED MEAN & SD

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

213

234

255

276

297

318

339Glucose (Jan 2011) with Manufacturer Mean & Ranges (SD – 21 ; CV- 7.6%)

Level 3

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

266.58

269.31

272.04

274.77

277.5

280.23

282.96

Glucose (Jan 2011) with Lab defined Mean & Ranges(SD – 2.73; CV- 0.99%)

Level 3

Quality Improvement : Monthly CV% Monitoring

                 

ASSAY Ricos CV%Target CV%

(Manufacturer) Jan Feb Mar Apr May Remarks

Lot No.                

                 

GLUCOSE 2.9 2.3 3.23 2.8 4.36 5.59 2.61 OK

CRTN 2.7 2.2 7.9 9.3 9.41 6.56 8.69 BAD

UREA 6.2 1.56 4.74 2.4 5.31 4.54 2.66 OK

CHOLESTEROL 2.7 3.84 3.84 2.7 5.86 4.13 2.66 OK

TRIGLYCERIDE 10.5 1.77 5.61 6.2 3.43 3.86 1.88 OK

HDL 3.6 3.11 2 5.3 8.87 3.76 2.87 Observe

SGPT 12.2 2.36 9.07 4.3 5.23 7.8 2.5 OK

SGOT 6 1.66 7.93 2.5 5.75 6.81 1.14 OK

ALKALINE 3.2 1.61 4.92 3 6.19 4.15 5.75 Observe

URIC ACID NA 1.94 3.87 1.4 1.51 3.05 2.97 Observe

                 

Remarks: Creatinine review and take corrective actions, validate if reqd

ACCURACY

Bias- measures how far your observed value is from a target value (peer group/consensus value)

Bias = Lab Value – True Value ; eg Bias is 10 if obtained value for Glucose is 110 & target is 100

Bias% = 10% for Glucose

SDI: Standard Deviation Index is a measurement of expressed as increments of SD

SDI = Your Mean - Peer Group Mean ……… ? how close your value is to the target value

Peer Group SD

SDI = 0.0 --- perfect - indicates no difference between lab mean and the consensus group mean

Z-score is the no. of standard deviations a control result is from the expected mean

Eg. Z-score of 2.3 observed value is 2.3 SD away from the expected mean

Total Error and TEa

Concepts of total error (TE) and total allowable error (TEa) - useful to choose SPC rules Total error (TE) = Bias% + CV% (Imprecision)

TE lab =Labbias (%) + 2.33 CVLab ……………………p<0/01

Six Sigma

- useful for quantifying test performance - In an ideal world, all our processes would be six sigma, and we could monitor them with very simple QC

Sigma = TEa - Bias / CV

Determine Quality Requirements for the Test: Biological variation information Clinicians' opinions, National and international expert bodies and agencies, Expert local groups or individuals Published professional recommendations eg Scandinavian Journal of Clinical and Laboratory Investigation

1999; 59: 585Performance goals set by: Regulatory bodies and agencies, Organizers of External Quality Assessment (EQA) schemes

Goals based on the current state of the art, including Inter-laboratory comparison programs, EQA or Proficiency Testing schemes

Frequency of QC run GLP - test normal and abnormal controls at least daily when

patient testing is performed. If the test is stable for less than 24 hours or some change has occurred which could potentially affect the test stability, controls should be assayed more frequently.

Eg, a diabetic patient in a critical care situation may have glucose levels run every 2 to 4 hours. In this case, it is important for the glucose test to be precise because lack of precision can cause loss of test reliability. If there is a lot of variability in the test performance (high imprecision, high standard deviation), the glucose result at different times may not be true.

NABL 112 GUIDELINES

W/L is <25/day --- 1 Level of QC

W/L is >25/day --- 2 Levels of QC

W/L is >75/day --- 2 Levels of QC -- twice a day at definite

intervals

*Min: at least one level of QC whenever patient samples are tested

Quality Controls Preparation, Use and QC Protocols:

As per manufacturer instructions in Kit insert

Storage and Stability of Controls

List of Parameters to be Analyzed with schedule of runs

• Multi control QC rules (WESTGARD RULES) given by Dr. James Westgard of the University of Wisconsin in an article in 1981 on laboratory quality control that set the basis for evaluating analytical run quality for medical laboratories.

• The Westgard system -based on the principles of statistical process control used in manufacturing nationwide since the 1950s

• Six basic rules in the Westgard scheme: 1-3s, 2-2s, R-4s, 1-2s, 4-1s, and 10x. These rules are used individually or in combination (multi-rule) to evaluate the quality of analytical runs.

• Detect random or systematic (shifts or trends) errors

• Warning 12SD or 1-2s:

It is violated if the single IQC

value exceeds the mean by 2SD.

Internal QC

• Rejection 22SD or 2-2s:

• This rule detects systematic error and is applied within and across runs.

• It is violated within the run when two consecutive control values exceed the "same" (mean + 2s or mean - 2s) limit.

• The rule is violated across runs when the previous value for a particular control level exceeds the "same" (mean + 2s or mean - 2s) limit.

Within run violation Across run violation

• Rejection 13SD or 1-3s:

• It is violated when the single IQC value exceeds the mean by 3SD.

• This rule is applied within control material only.

• The 1-3s rule identifies unacceptable random error or possibly the beginning of a large systematic error.

• Rejection 41SD or 4-1s:

It is violated if four consecutive IQC

values exceed the same mean plus 1s

or the same mean minus 1s control limit.

• Rejection 10x:

• This rule detects systematic bias and is applied both within and

across control materials. • It is violated across control materials if the last 10 consecutive

values, regardless of control level, are on the same side of the

mean.• The rule is violated within the control materials if the last 10 values

for the same control level are on the same side of the mean.

Multi Control QC Rules• The multi control QC rules are followed in the department as described

below:

The rules to follow when one level QC material is used:• Reject QC if:i) it is outside 3 SD (13s)ii) two consecutive values obtained are outside 2 SD on the same side but within 3 SD (22s)iii) ten consecutive values are above or below the mean, but within 2 SD (10x)

The rules to follow when 2 level QC materials are used:• Reject QC if:i) either QC values is outside 3 SD (13s)ii) both QC values are outside 2 SD on the same side, but within 3 SD (22s)iii) difference between both QC values is >4 SD i.e. one level QC is > 2 SD and other level QC is <2SD

(R4s).iv) ten consecutive values of the same level QC are >/< the mean, but within 2 SD (10x).v) five consecutive values of one level QC and five consecutive values of other level QC are >/< the

mean but within 2 SD (10x)

Westgard Procedure Flowchart

SYSTEMATIC VS. RANDOM ERRORS

Systematic ErrorAvoidable error due to controllable variables in a measurement.

Random ErrorsUnavoidable errors that are always present in any measurement. Impossible to eliminate

Review to Manage “Out of Control” Situation:

Random Errors:

Search for recent events/changes:

New reagent kit or lot New control bottle Instrumentation component

replacement Instrument maintenance Instrument move

Search for sources of Random Error: Power supply Double pipetting of control sample Misplacement of control sample within

the run Air bubbles in water supply Random air bubbles in reagent or sample

pipette system Incorrect reconstitution of the control

product Inappropriate storage of control in frost

free freezers Use of non-reagent grade water in the

test system Operator technique

Sources of Systematic Errors:

Improper alignment of sample or reagent pipettes

Drift or Shift in incubator chamber temperature

Inappropriate temperature / humidity levels in the

testing area

Change of reagent or calibrator lot

Deterioration of reagent while in use, storage or

shipment

Deterioration of calibrator or control product

while in use, storage or shipment.

Incorrect handling of control product (e.g.

Freezing when not recommended)

Inappropriate storage of control products in frost

free freezers

Failing light source

Use of non-reagent grade water in the test system

Recent calibration

Change in test operator

Specimen carry-over

Obstruction of tubing

Corrective Actions• Classify error, inform n document • Review error for cause – detailed• Relate error to cause• Rerun QC/retained sample• Follow manufacturer’

troubleshooting guide• Run fresh controls if reqd• Call for application’s support if…• Take appropriate CAPAs actions

and document the details in relevant “QC Failure Log”

EXTERNAL PERFORMANCE TESTING

Blind samples submitted to laboratories Labs must periodically analyze in order to assure

Quality for acceptable results Accreditation Reqts – Mandatory Cl 5.6

??

?

PROFICIENCY TESTING/EQAS Process the PT samples in the same run as patient samples

Process as per the instructions given along with the PT sample & not in duplicate

Don’t send for analysis to another or referral laboratory

Do n’t discuss the results with another laboratory till the evaluation reports from the PT provider are available

EVALUATION OF PT PERFORMANCE:

Review Unsatisfactory PT result for remedial corrective actions to be taken. See if there are any trends or shifts.

Review Tips : Check for

Clerical errors & operator investigation

IQC data for the date PT samples were run, stability, expiry, peer group Z score, trends ans shifts etc

Reagent logs – expiry, onboard stability, contamination, suitable calibration for new lot

Calibration logs – cal curve, schedule to determine how close to the PT samples the calibration was performed

Equipment logs- maintenance performed

Rerun frozen PT sample if available Once the problem that caused the PT failure is found, it is rectified, documented and reviewed

with laboratory staff All un-graded PT surveys are investigated to the acceptability of its performance with the same

rigor as if it were an unacceptable performance followed by corrective action and documentation if required

All the participant statistics are reviewed and documented

QA --- OTHER PRACTICES IN LAB

QC Lot to Lot validation: Establishing Lab Reference Ranges with change in QC Lot

Assayed QC: Compare for two days old lot with new lot Unassayed QC: Do as new till then data points are evaluated New Test Validation Validation of Calculated Parameter Proficiency Testing/EQAS CALIBRATION:

As indicated by calibration frequency When Quality Control is outside Range New lot of Reagent Introduced New lot of Control is used New company QC reagent is used

Reagent/Kit validation Whenever reagent/kit, Kit lot no changes, run controls and/or patient

samples which have been previously tested by the old reagent/kit to detect any significant changes

Accept if within 2SD. If unacceptable, then repeat in duplicate and record SD/CV

FINAL REPORTING

EACH AND EVERY REPORT MUST BE SCRUTINIZED

AND AUTHENTICATED BY DESIGNATED PERSONNEL

ONLY

50

EVERY LABORATORIAN HAS A RESPONSIBILITY AND AN OBLIGATION TO BE INFORMED ABOUT QUALITY AND ITS MANAGEMENT.

THANK YOU !