eqas interpretation

8/12/2019 EQAS Interpretation

1/69

Many lab professionals think Qualityin a Medical Laboratory is .

Accurate

Timely

Reliable

Reproducible

RESULTS


2/69

INSTITUTE OF BIOCHEMISTRY WELCOMES YOU ALL TO

THE CME

ON

QUALITY ASSURANCE IN CLINICAL

LABORATORY

UP, CLOSE & PERSONAL


3/69

8 rights make Quality in a MedicalLaboratory is .

Choosing the Righttest

Collecting RightSpecimen

After Rightpatient preparation

Testing by the Rightmethod

Reporting the Rightbased on

RightReference intervals at the

Righttime and at the

RightPrice


4/69

Tissue

Material

Suppliers

Equipment

Process or

Techniques

Environment

Personnel

Patient or Client

(sample sources)

Physical Damage

or Contamination

Microbial

Contamination

Result:

Accuracy

Precision

Reliability

EfficiencyConfidentiality

Information

MrX

MsY

Where can you go wrong in a lab ?


5/69

Quality is .

A subjective term - for which each person has his / her owndefinition

Technically Quality can have two meanings

A product or service that fulfills the defined and expected

requirement

stated and implied needs

A service or product free from defects & deficiencies


6/69

QUALITY CONTROL

QUALITY ASSURANCE

QUALITY SYSTEM

QUALITY

MANAGEMENT

Stages of Quality - Hierarchy


7/69

WHO definition of QA & QC is .

QA - Includes Internal QC, External QA, pre-analyticphase, test standardization, post-analytic phase,management, and organization (WHO, 1992)

QC - Internal quality control (IQC)set of proceduresfor continuously assessing laboratory work and theemergent results; immediate effect, should actuallycontrol release of results (WHO, 1981)


8/69

Quality Assurance & QualityControl - difference is .

QA is for correction & prevention of errors or defectsin the entire lab

QC is detection of errors and defects in testing process


9/69

Purpose of Internal & External QC

Internal QC

For CONTINUOUS & IMMEDIATE (DAILY)monitoringof the laboratory work and the emergent results in order todecide whether the results are reliable enough to be releasedto physicians(WHO, 1981)

Measures Precision& Repeatabilityof the systems &methods in use in the lab.

Illusion of short term accuracy of the lab results.

Do not detect the accuracy or trueness of patient results over alonger term


10/69

Purpose of Internal & External QC

External QC

For PERIODIC AND RETROSPECTIVE monitoring of lab

results by an independent external agency to indicate to thelaboratory and its staff the accuracy or bias in their systems &methods

Lab can know its shortcomings and change their Internal QualityAssurance procedures.


11/69

Why EQAS is necessary?Serves as an educational tool and help to monitor & improve the

performance of the lab

Measures the accuracy or bias of its results and stability ofmethodsOver a longer period of time in terms of years

Mandatory requirement for applicant & accredited labs

Non participation or repeated failures in an EQAS or PT

programme may result in temporary suspension or cancellation ofaccreditation for those non EQAS tests

Gives the laboratories, both the management & technical staff,

added confidence in their patient test results


12/69

WHAT DOES IT IDENTIFY ?

Identifies systematic kit & reagent problems, water qualityproblems, analyte calibration stability and status, equipmentperformance

Indicator of where to direct improvement efforts

Identifies training needs of lab personnel

Benchmarks the labs performance against others

Early Warnings System for Problems


13/69

HOW SHOULD IT BE USED ?

Should only be used for motivating staff & not topunish them

Inaccurate lab results are not due to technicians

But due to failure of lab systems


14/69

HOW SHOULD IT BE USED ?

EQAS / PT samples should be treated exactly as the patientsamples Only then the correct situation in the lab can be foundfixing the problem becomes easy

Never

Run the calibration on the day of reporting EQAS sample if it isnot a scheduled /required calibration

Repeat the EQAS samples where as the patient samples are testedonly once and give the mean of multiple runs

Ask a specific analyst run the EQAS / PT sample


15/69

REMEMBEREQAS.

SUPPLEMENTS Internal Quality Control

NEVER a SUBSTITUTE for Internal QC

Both measure 2 different aspects of quality


16/69

INTERPRETATION OF EQAS REPORTS

1.VISVariance Index Score

2. SDIStandard Deviation Index

3. Z-ScoreClassical & Robust


17/69

Variance Index Score

First proposed by the United Kingdom National Quality ControlScheme (UKNEQUAS)

CCV (Chosen Co-efficient of Variation) & DV (Designated Value)

used to calculate VIS

CCV is just the Allowable Limit of Error for an analyte (TEa) - Sumof both imprecision and bias

Set & recommended by WHO after studying the performance ofmany Indian labs


18/69

CCV of some common Analytes

Glucose 7.5 Sodium 2.3

Urea 10 Potassium 5.0Creatinine 10 Chloride 6.0

CK 7.3 AST 12.5

T.Bilirubin 19.2 ALT 17.3

T.Protein 7.5 ALP 15.5

Albumin 7.5 Amylase 15.5

Calcium 6.0 LDH 15.5

Uric acid 7.7 Phosphorus 7.8Cholesterol 7.5 Bicarbonate 9.0

TGL 14 HDL- C 7.6

HDL 7.6 Iron 15


19/69

Calculation of VIS

Designated Value [DV] = 120 mg %

Participant's result = 95 mg%

% Variation [%V] = Participant's Result - Designated value--------------------------------------- X 100

Designated value

120-95 X 100 = 25 X 100120 120

= 20.8VarianceIndex = %V X 100 = 20.8 X 100 = 277

CCV 7.5VIS = 277


20/69

How to read the EQAS results ?

Check the VIS & OMVIS values for each parameter every month

Check if your value is close to DV

Closer it is lower will be your VIS & better is your labs accuracy

Remember If your VIS is < 50 it is regarded and given as zero score

Even if >400, it is still given as 400 only

Calculation of VIS


21/69

Interpretation of VIS

VIS Performance

< 100 Very good

100 -150 good

150 -200 satisfactory room for improvement

> 200 Not acceptable

If VIS of >200 on two or more occasions for the same analyte,them check your standardization procedures & calibration

Indicates an accuracy problem (systematic error / bias )


22/69

Interpretation of VIS

Check the monthly OMVIS.(Overall Mean VIS) cumulativeperformance over a period

OMVIS Performance

< 100 Very good - your result are very close to DV

150-200 Need to take care of those parameters for whichthe reported values are very different from the DV for that particularmethod

> 250 You are probably reporting many wrong results &you should take urgent steps to locate the problem and correct them


23/69

Calculation of VIS Method MeanThe Method Mean' - Mean obtained from results of allparticipating labs following the same method including results ofoutliers

The Designated Valueis the value obtained after excludingresults, from labs with same method, which are > 3SD of MethodMean and recalculating the mean after eliminating the outliers - Mean

of inliers only

The 'Reference Mean' - Mean obtained at the organizing lab afterexposing the QC samples to ambient temperature (25-35 C) for a

period of 7- days (transport time) and analysing them on five differentdays

The reference mean is shown against the method by which it wasanalysed in the organizing lab


24/69

Mean value obtained atthe Reference Lab (CMC)after exposing 5 vials toambient Temp. for 9days and analyzing themon 5 diff. days


25/69

Mean value obtained at theReference Lab (CMC) afterexposing 5 vials to ambientTemp. for 9 days and analyzing

them on 5 diff. days


26/69

Mean value of results fromof all participating labs withsame method


27/69

Value given (DV) is the mean of allparticipating labs for that method afterexcluding results from labs outside 3SDof the Mean Value (of the participatinglabs with the same method)


28/69

STANDARD DEVIATION INDEX

Another Statistical toolassigned to the lab bythe EQAS / PT provider on the performance ofthe lab for each analyte in a EQAS cycle

A measure of relativeinaccuracy / relativebias

Wh t i l G i


29/69

What is normal or Gaussiandistribution ?

Out of 100 results

68.2% of values fall

within

1SD


within 2SD


within 3SD


30/69

IS KNOWING SDI USEFUL ?

Yes - A measure of the result around a mean among agroup of values

Since 95 % of all results in a normal population fallwithin 2 SDs of the mean, +2 SD is considered anacceptable laboratory value

Expressed in the units being measured


31/69


The number of Standard Deviations that your labs mean differsfrom the Peer Group Mean

. Difference is converted into SD units (SDI)

The SDI indicates how large / small the difference is betweenyour result and target value

Simply said

Difference between your result and group mean in

terms of the number of standard deviations from the

overall mean


32/69


Actual magnitude of the difference in the units of thetest may look too small or too large

To figure the actual size of this inaccuracy / bias inconcentration units, you need to multiply by the actualSDI by of the group SD.

For e.g if the group mean is 102 mg/dL for TGL and

group SD is 5 mg/dL and your SDI score is 1.2Actual quantitative difference is 1.2 x 5 = 6 mg/dL


33/69


Any SDI of 2.0 or greaterin a EQAS cycle for anyanalyte deserves special concernindicates some for ofsystematic error

Any test whose average SDI is 1.0 or greater deserves

some special attention because your method shows asystematic difference from the group.

Likely to lead to unacceptable results in future

SDI up to 1.0your performance is satisfactoryyourresult is with 1 SD of the groupyou are with in the68.7 % of labs result whose values are close to mean


34/69


if you observe SDIs such as -0.4, -0.2,-0.5 - 0.5, - 0.5 and -1.0 (all negative)for an analyte in successive cycles,your method is generally running onthe low side and is negatively biased,

on average, by +0.6 SDI

You are reporting precise pateintvalues but lower than the true valueby 0.6 SD ()

So better

calibrate your instrument and analyte

or requires instrument maintenance


35/69

Z-Score

Classical Z Scoresame as SDICan be used for internal quality control also


36/69

Z-Score

Robust Z scorestatistic is used when the distribution of resultsof participating labs is not Gaussian (not normally distributed)and there are varied results / outliers

Both accuracy and precision (repeatability as well as

reproducibility) are assessed in terms of robust Z score - bothwithin and between labs Z score (ZB & ZW)

The participant labs are asked to analyse the same sample

TWICEand submit both results to the EQAS provider


37/69

Z-Score


38/69

Z-ScoreRobust Z score =

Normalised Labs result- Median result of all labs )

Normalized IQR ( Inter Quartile Range)

It is calculated based on the median value (central value) and theinterquartilerangeAll results from participating labs are arranged in an ascending manner (lowestto highest) the central value is taken as the median

The 25th and 75th percentile values are calculated The interquartile range is thedifference between the 75th & 25th percentile

The 25th quartile (Q1) is the value below which a quarter of the results lie.Similarly, the 75th quartile (Q3) is the value above which a quarter of the resultslie.

IQR = Q3 - Q1

Normalized inter quartile range (NIQR): NIQR = IQR x 0.7413 (a

constant)


39/69

Z-Score

The between laboratory Z-score (ZB) =(S - median ) / (IQR x 0.7413)S = (A + B)/[square root of (2)] = standardised sum ofthe two results for a laboratory (where A and B areresults of two samples of the same test).

Within laboratory Z-score (ZW) =

(D- median / (IQR X 0.7413)D = (A - B)/[square root of (2)] = standardized

difference between the two results for a laboratory Whiletesting two specimens in an EQAS / PT programme bya lab performance both ZW and ZB should beconsidered simultaneously for assessing the performance


40/69

Z-Score

Interpretation of Robust Z-scores:Z score less than 2 - SatisfactoryZ score 2 but less than 3 - Questionable perfromanceZ score more than 3Unsatisfactory

Both ZB & ZW should be < 2

Using only one Z-score may misleading

ZW < 2 ZB > 2 = Higher bias i.e low reproducibility

ZB < 2 ZW > 2 = low precision (i.e., low repeatability)

For assessing laboratory's technical competency, bothZW and ZB value should be low at the same time.


41/69

EXERCISES ON INTERPRETATIONOF EQAS REPORTS

Dr.V.K.Ramadesikan


42/69

Why Analysis of EQAS reports is important ?

True benefit of EQAS /PT proficiency testing, lies incarefully & critically evaluating your results and report

Proper analysis of EQAS testing results can revealproblems even before failure in EQAS or even an

adverse patient resultPotential problems can be recognised by recognisingpatterns from graphs

Review both the present cycle results as well as

performance from previous cycle for the same analyteGraphs of Z score or SDI or VIS or % deviation fortrends

Otherwise trends will be missed


43/69

How to recognise problems ?

Results may consistently be different from thetarget peer group mean - All results on one sideof the mean (may be close to mean or at variabledistances from mean)SystematicError

Majority of results are close to the target valuebut some show larger deviation s on one side orboth side of meanRandomError

Reasons and hence corrective action differs


44/69

How to recognise problems ?Random error is an error / mistake / inaccuracy that has no set

pattern. Its occurrence cannot be predicted

Results on an average are close to target mean. Few results showlarge deviations on either side of target

Detected by sudden, undue % deviation /SDI /Z Score> 3.5

Indicates labs imprecision / poor reproducibility

Easily identifiedvalues are far beyond the usuale.g SDI from1.5 to 4.06.0


45/69

How to recognise problems ?

Systematic errorset pattern of error / mistake. Its occurrencecan be explained and corrected

Constant difference between the participant labs value and

group mean All results lie on one side of mean

Indicates labs bias or inaccuracy but good precision

A progressive increase in deviation on the same side shift orstabilizes after a gradual increase trends in


46/69

Sources of Random Errors

Random errors are blunders

Transcription errorsresult not correctly transcribed from theinstrument to workbook or PT sheet or from the Workbook to thesystem

Misplaced decimal points e.g serum potassium 58.2 instead of 5.82

Result was entered in the wrong instrument or method group on the

result form of PT provider.

Lab might have changed the method or instrument recently but notupdated with PT provider


47/69

Sources of Random Errors

Mislabeling errors, interchanging results of PT specimens,misplacing specimens in analyser rack, inappropriate reagents andstandards

Result of some other analyte entered in the PT form of the provider

in the system

Wrong units. Result in one unit in the instrument but not convertedto unit of EQAS provider e.g ug /mL instead of ng/L , mg/dLinstead of g/L

Result found on evaluation report from PT provider not matchingwith the result on the report form mistake of PT provider

Calculation errors (conversion from one unit to another)


48/69

Sources of Systematic ErrorsRecalibration if not done earlieresp if a new lot of reagent has

been used or if the open vial stability of the reagent is doubtful

Instrument maintenancemajor part needs servicing orreplacement (optics, alignment, incubation temperature failure,Internal QC values having a bias

Recent instrument malfunction, instrument failure soon after PTspecimen testing

Assay settings (sample volume, reagent volume, delay time

incubation time no of readings to be taken etc. )

improper reconstitution of QC materials, water quality, notfollowing manufacturers / PT providers instructions whilereconstituting, storing or handling reagents / QC materials

What type of Error is indicated


49/69

What type of Error is indicatedin greenand red?


50/69

Systematic Error

SAMPLE EQAS MONTHLY REPORT


51/69


Your result



52/69


SAMPLE EQAS END OF CYCLE REPORT


53/69




54/69



55/69

SAMPLE CAP EQAS REPORT


56/69




57/69




58/69


SAMPLE EQAS REPORT


59/69

SAMPLE EQAS REPORT

Transcription errorMisplacing of specimens or interchanging resultsof PT specimens

U 1 31.3 909.51 U3

U 2 13.1 28.31 U1

U 3 1031.9 14.8 U2


60/69

Result - 226 Mean445 SDI 4.34

ALT226 U/L

Mean445

SDI

4.34


61/69

Analytical errorCalculation error

PT was repeated with 1 in 2 dilution as results were

high but the result was not multiplied by 2

Diluted result was sent

SAMPLE EQAS REPORT


62/69

SAMPLE EQAS REPORT

SAMPLE EQAS REPORT


63/69

SAMPLE EQAS REPORT

Erratic ResultsVery poor precision and accuracy

ProblemInternal Controls wide SD due to variousreasons

Instrument not maintained & calibrated reagentsproblems deterioration

Analyte not calibrated

Procedural problems



64/69


SAMPLE EQAS REPORT


65/69

SAMPLE EQAS REPORT

1. Systemic error may have seeped in the system

anytime after the last EQAS sample reportinglook at daily controls for any shift in values

2. May be a random errorKeep a watch on dailycontrols and follow during next EQAS

3. Wrong sample was tested

4. Wrong preanalytical stagereconstitution,storage etclook at values of other analytes

Are they with in acceptable limits of SDI and in the

same direction ?



66/69


SAMPLE EQAS REPORT


67/69

SAMPLE EQAS REPORT

1. Systematic errorone showing a positive bias

and other a negative bias

2. Needs

Recalibration for analyte

Instrument maintenance

Internal QC Value Mean needs to be reset

SAMPLE EQAS REPORT


68/69

SAMPLE EQAS REPORT

SAMPLE EQAS REPORT


69/69

SAMPLE EQAS REPORT

1. Fairly a good performance in terms of accuracy

and precision

2. Values on either side of mean No Bias

3. Values within =/- 0.5 SD

eqas interpretation

Documents