influence of analytical bias and imprecision on guideline-driven medical decision limits per hyltoft...

Influence of Analytical Bias and Imprecision on

Guideline-Driven Medical Decision Limits

Per Hyltoft Petersen

Hyltoft Petersen P, Klee GG.

Influence of Analytical Bias and Imprecision on the Number of False Positive Results Using Guideline-Driven Medical Decision Limits.

Clin Chem Acta 2014;430:1-8

LABMED 2014

“Guideline-Driven Medical Decision Limits”.

Diagnostic decisions based on decision limits according to medical guidelines are different from the majority of clinical decisions

due to the strict dichotomization of patients into diseased and non-diseased based on the biochemical measurement of a single component.

Background:

Consequently, the influence of analytical performance is more critical than for other diagnostic decisions where many other informations are included in the diagnosis.

Cholesterol as a screening-test for preventing coronary heart disease in adults

Decision: Cholesterol above or below 6.2 mmol/L


HbA1c in diagnosis of diabetes mellitus

Decision: HbA1c above or below 48 mmol/mol (6.5 % HbA1c)

Examples:

Sacks et al. Diabetes Care 2011;34:c61-c99 Garber et al. Ann Intern Med.. 1996;124:515-517


HbA1c reference interval for healthy

According to traditional IFCC criteria

CVWITHIN-SUBJECT = 1.94 % in IFCC units (~ ln = 0.0194)

HbA1c:

Recommended cut-off = 48 mmol/mol ~ ln = 3.86

Jørgensen et al. Scand J Clin Lab Invest 2002; 62:609-22.

Carlsen et al. Clin Chem Lab Med 2011;49:1501-7

Log-Gaussian distribution (natural logarithm) ln-mean = 1.727 and ln-standard deviation = 0.053


Distribution of healthy set-points

CVWITHIN-SUBJECT = 1.94 %

HbA1c:

Distribution of Set-P oints of Reference Individuals and P robability of Results Above Cut-Off for one Sampling

5030 35 40 45

0.0

0.2

0.4

0.6

0.8

1.0

3.4 3.5 3.6 3.7 3.8 3.9 4.0

HbA1c, mmol/mol

Fre

qu

en

cy o

r P

rob

ab

ility

Distribution of Set-Points

HbA1c ln(mmol/mol)

Cut-off = 48 mmol/mol

Distribution of values from a person with set-point = cut-off


5030 35 40 45

0.0

0.2

0.4

0.6

0.8

1.0

3.4 3.5 3.6 3.7 3.8 3.9 4.0

HbA1c, mmol/mol

Fre

qu

en

cy o

r P

rob

ab

ility


HbA1c ln(mmol/mol)



HbA1c:

HbA1c: Cut-off = 48 mmol/mol with Frequency and Probability due to Within-subject variation

Cut-off

Cut-off

5251504948474645

0

0.2

0.4

0.6

0.8

1

3.80 3.84 3.88 3.92 3.96

HbA1c, ln(mmol/mol)

Fre

qu

ency

or

Pro

bab

ility

frequency

cumulated

Cut-off

HbA1c, mmol/mol

Probability

Frequency

Distribution for a person with set-point = cut-off

Frequency Cumulated frequency (probability)

Probability of being measured above cut-off50 %

50 %

HbA1c: Cut-off = 48 mmol/mol with Probability for

one and two samplings

Cut-off

45 46 47 48 49 50 51 52

0

0.2

0.4

0.6

0.8

1

3.80 3.84 3.88 3.92 3.96

HbA1c, ln(mmol/mol)

Pro

ba

bil

ity

One sampling

Two samplings

Cut-off

HbA1c, mmol/mol

One sampling Two samplings

25 %

50 %


HbA1c:


Effect of one and two samplings

HbA1c: Cut-off = 48 mmol/mol with Probability for

two Samplings with Imprecision = 0, 2 and 4 %

Cut-off

5251504948474645

0

0.2

0.4

0.6

0.8

1

3.80 3.84 3.88 3.92 3.96

HbA1c, ln(mmol/mol)

Pro

bab

ility

Imprecision 0%

Imprecision 2%

Imprecision 4%

Cut-off

HbA1c, mmol/mol

Unbiased

2 %

Imprecision

4 %0 %


HbA1c:

Distribution for two samplings with set-point = cut-off

Effect of imprecision

HbA1c: Apparant Cut-off with Probability for varying bias ± 4 % and Imprecision 0 % for two Samplings

5251504948474645

0

0.2

0.4

0.6

0.8

1

3.80 3.84 3.88 3.92 3.96

HbA1c, ln(mmol/mol)

Pro

bab

ility

Unbiased

+4 % bias

-4 % bias

Cut-off

HbA1c, mmol/mol

Unbiased

Apparant Cut-off for Bias = + 4 %

Apparant Cut-off for Bias = - 4 %


HbA1c:

Distribution for two samplings with set-point = cut-off

Effect of Bias


Distribution of healthy set-points

CVWITHIN-SUBJECT = 1.94 %

HbA1c:


5030 35 40 45

0.0

0.2

0.4

0.6

0.8

1.0

3.4 3.5 3.6 3.7 3.8 3.9 4.0

HbA1c, mmol/mol

Fre

qu

en

cy o

r P

rob

ab

ility


HbA1c ln(mmol/mol)



45403530 50

0.0

0.2

0.4

0.6

0.8

1.0

3.4 3.5 3.6 3.7 3.8 3.9 4.0

HbA1c, mmol/mol

Fre

qu

en

cy o

r P

rob

ab

ility


Probability of >cut-off

HbA1c ln(mmol/mol)


Probability for a person with set-point = cut-off to be measured above cut-off

Distribution of Set-Points of Reference Individuals and Probability of Results Above Cut-Off for one Sampling

5030 35 40 45

0.0

0.2

0.4

0.6

0.8

1.0

3.4 3.5 3.6 3.7 3.8 3.9 4.0

HbA1c, mmol/mol

Freq

uenc

y or

Pro

babi

lity Set-point distribution

Cut-off = 3.86 = 48CVAnalytical = 0 %CVAnalytical = 1 %CVAnalytical = 2 %CVAnalytical = 3 %CVAnalytical = 4 %CVAnalytical = 5 %Mean = 3.63 = 38CVBetween = 8.38%CVWithin = 1.94%Bias % = 0.0

Distribution of Set-Points Prob of

>cut-off

HbA1c ln(mmol/mol)



HbA1c:

Distribution of set-points (healthy population)

Cumulated frequency (probability functions) One sampling

Increasing imprecision

Distribution of Set-Points of Reference Individuals and Probability of Results Above Cut-Off for one Sampling

5030 35 40 45

0.0

0.2

0.4

0.6

0.8

1.0

3.4 3.5 3.6 3.7 3.8 3.9 4.0

HbA1c, mmol/mol

Freq

uenc

y or

Pro

babi

lity Set-point distribution

Cut-off = 3.86 = 48CVAnalytical = 0 %CVAnalytical = 1 %CVAnalytical = 2 %CVAnalytical = 3 %CVAnalytical = 4 %CVAnalytical = 5 %Mean = 3.63 = 38CVBetween = 8.38%CVWithin = 1.94%Bias % = 0.0

Distribution of Set-Points Prob of

>cut-off

HbA1c ln(mmol/mol)



HbA1c:

Distribution of set-points (healthy population)


Now we take a sample within a small interval of healthy set-pointsand multiply with the probability of these set-points exceed the cut-offto get the FP for healthy with this set-point

and by repeating the process for all intervals we get the distribution of origins of set-points of FP

Origin of set-Points Measured Above Cut-Off forno Bias and Varying Imprecision for one Sampling

5045403530

0.0000

0.0001

0.0002

0.0003

0.0004

0.0005

3.4 3.5 3.6 3.7 3.8 3.9 4.0

HbA1c,mmol/mol

Fre

qu

en

cy

Cut-off = 3.86 = 48

CVAnalytical = 0 %CVAnalytical = 1 %



Mean = 3.63 = 38CVBetw een = 8.38%

CVWithin = 1.94%Bias % = 0.0

Origin of Set-Points for varying imprecision

HbA1c ln(mmol/mol)



HbA1c: Varying imprecision and bias

Bias = 0 %

Bias = + 4 %

The effect of positive bias is like moving the cut-off to the left


5045403530

0.0000

0.0001

0.0002

0.0003

0.0004

0.0005

3.4 3.5 3.6 3.7 3.8 3.9 4.0

HbA1c,mmol/mol

Fre

qu

en

cy

Cut-off = 3.86 = 48




Mean = 3.63 = 38CVBetw een = 8.38%



HbA1c ln(mmol/mol)


Origin of set-Points Measured Above Cut-Off for Bias = + 4 % and Varying Imprecision for one sampling

5030 35 40 45

0.0000

0.0001

0.0002

0.0003

0.0004

0.00053.4 3.5 3.6 3.7 3.8 3.9 4.0

HbA1c mmol/mol

Fre

qu

en

cy

Cut-off = 3.83 = 46.1CVAnalytical = 0 %


CVAnalytical = 3 %CVAnalytical = 4 %CVAnalytical = 5 %

Mean = 3.63 = 38CVBetw een = 8.38%

CVWithin = 1.94%Bias % = + 4

Origin of Set-Points for Bias = + 4 % and varying imprecision and bias = + 4 %

HbA1c ln(mmol/mol)

Apparant Cut-off for Bias = +4 %

Origin of set-points for healthy individuals measured above 48 mmol/mol (healthy diagnosed as diabetics)


HbA1c: Varying imprecision and bias

Bias = 0 %

Bias = + 4 %Origin of set-Points Measured Above Cut-Off for

Bias = + 4 % and Varying Imprecision for two samplings

50454035

0.00000

0.00010

0.00020

0.00030

0.00040

0.00050

3.4 3.5 3.6 3.7 3.8 3.9 4.0

HbA1c mmol/mol

Fre

qu

en

cy

Cut-off = 3.83 = 46.1

CVAnalytical = 0 %

CVAnalytical = 1 %

CVAnalytical = 2 %

CVAnalytical = 3 %

CVAnalytical = 4 %

CVAnalytical = 5 %

Mean = 3.63 = 38

CVBetw een = 8.38%

CVWithin = 1.94%

Bias % = + 4

HbA1c ln(mmol/mol)


Origin of Set-Points for Bias = + 4 % and varying imprecision

Bias = + 4 %

One Sampling

Two Samplings

One and two samplings

One Sampling

Origin of set-Points Measured Above Cut-Off for Bias = + 4 % and Varying Imprecision for one sampling

5030 35 40 45

0.0000

0.0001

0.0002

0.0003

0.0004

0.00053.4 3.5 3.6 3.7 3.8 3.9 4.0

HbA1c mmol/mol

Fre

qu

en

cy

Cut-off = 3.83 = 46.1CVAnalytical = 0 %


CVAnalytical = 3 %CVAnalytical = 4 %CVAnalytical = 5 %

Mean = 3.63 = 38CVBetw een = 8.38%

CVWithin = 1.94%Bias % = + 4

Origin of Set-Points for Bias = + 4 % and varying imprecision and bias = + 4 %

HbA1c ln(mmol/mol)



5045403530

0.0000

0.0001

0.0002

0.0003

0.0004

0.0005

3.4 3.5 3.6 3.7 3.8 3.9 4.0

HbA1c,mmol/mol

Fre

qu

en

cy

Cut-off = 3.86 = 48




Mean = 3.63 = 38CVBetw een = 8.38%



HbA1c ln(mmol/mol)


Origin of set-points for healthy individuals measured above 48 mmol/mol (healthy diagnosed as diabetics)


HbA1c:

Percentage of healthy individuals measured > 48 mmol/mol (false positive diabetics)

As function of bias % for varying percentages of imprecision

Inf luence of Bias and Imprecision on Percentage Reference Individuals w ith HbA1c ≥ 48 mmol/mol in one Sample

Bias = 0%

0

2

4

6

-10 -8 -6 -4 -2 0 2 4 6 8 10Bias %

Per

cent

age

FP

CVA % = 0CVA % = 1CVA % = 2CVA % = 3CVA % = 4CVA % = 5CVA % = 6CVA % = 7CVA % = 8CVA % = 9CVA % = 10

For one sampling


HbA1c: As function of bias % for varying percentages of imprecision

Inf luence of Bias and Imprecision on Percentage Reference Individuals w ith HbA1c ≥ 48 mmol/mol in one Sample

Bias = 0%

0

2

4

6

-10 -8 -6 -4 -2 0 2 4 6 8 10Bias %

Per

cent

age

FP


For one sampling

Inf luence of Bias and Imprecision on Percentage Reference Individuals w ith HbA1c ≥ 48 mmol/mol in tw o Samples

Bias = 0%

0

2

4

6

-10 -8 -6 -4 -2 0 2 4 6 8 10Bias %

Per

cent

age

FP


For two samplings

Percentage of healthy individuals measured > 48 mmol/mol false positive diabetics

HbA1c:

Inf luence of Bias and Imprecision on Percentage Reference Individuals w ith HbA1c ≥ 48 mmol/mol in tw o Samples

Bias = 0%

0

2

4

6

-10 -8 -6 -4 -2 0 2 4 6 8 10Bias %

Per

cent

age

FP


What are the recommended quality specifications from What are the recommended quality specifications from Sacks et al. Clin Chem 2011;57:793-8 Desirable specifications for HbA1c measurement are an Desirable specifications for HbA1c measurement are an intralaboratory CV < 2% and an interlaboratory CV < 3.5 % intralaboratory CV < 2% and an interlaboratory CV < 3.5 %

The CV 3.5 % DCCT units corresponds to 5.2 % at 48 The CV 3.5 % DCCT units corresponds to 5.2 % at 48 mmol/mol in IFCC units, and reduced by the 2 %, the final mmol/mol in IFCC units, and reduced by the 2 %, the final allowable bias is from ± 9 % at a 95 % intervalallowable bias is from ± 9 % at a 95 % intervaland false positives could be from 0 to 2.8 % and false positives could be from 0 to 2.8 %

Personal information from Personal information from Thomas Røraas and Sverre Sandberg, Thomas Røraas and Sverre Sandberg, NOKLUS, Bergen, NorwayNOKLUS, Bergen, Norway


There is no reference interval for Cholesterol due to the strict decision limit of 6.2 mmol/L

95 % limits 150-275 mg/dL = 3.89-7.12 mmol/L

Log-Gaussian distribution (natural logarithm)

CVTOTAL =15.2 % ~ ln = 0.152

CVWITHIN-SUBJECT = 6.0 % ~ ln = 0.060

CVBETWEEN-SUBJECT = 13.9 % ~ ln = 0.139

Recommended cut-off = 6.2 mmol/L ~ ln = 1.825

Ricos et al. Scand J Clin Lab Invest 1999;59:491

Cholesterol:

But a range for the total population can be estimated

Klee et al. Scand J Clin Lab Invest 1999;59:509


Cholesterol:


Effect of one and two samplings

Cholesterol: Probability for Treatment with one and two Samplings as Function of Set-point

Cut-off

8.07.57.06.56.05.55.0

0.0

0.2

0.4

0.6

0.8

1.0

1.60 1.70 1.80 1.90 2.00 2.10

Cholesterol, ln(mmol/L)

Pro

bab

ility

One sampling

Two samplings

Cut-off

Cholesterol Set-point, mmol/L

One sampling Two samplings

25 %

50 %

Distribution of Set-Points of Low Risk Individuals and Probability of Results Above Cut-Off - No Bias

4.0 5.0 6.0 7.0 8.0

0.0

0.2

0.4

0.6

0.8

1.01.3 1.4 1.5 1.6 1.7 1.8 1.9 2.0 2.1

Cholesterol, mmol/L

Fre

quen

cy o

r P

roba

bilit

y Set-point distributionCut-off = 1.8262 = 6.21CVAnalytical = 0 %CVAnalytical = 2 %CVAnalytical = 4 %CVAnalytical = 6 %CVAnalytical = 8 %CVAnalytical = 10 %Mean = 1.66 = 5CVBetween = 13.90%CVWithin = 6.00%Bias % = 0.0

Distribution of Low Risk Set-Points Probability of

>cut-off

ln(cholesterol)

"Cut-off" = 6.21 mmol/L


Cholesterol:

Distribution for persons with set-points below 6.2 mmol/L



Cholesterol:


Cumulated frequency (probability functions)


4.0 5.0 6.0 7.0 8.0

0.0

0.2

0.4

0.6

0.8

1.01.3 1.4 1.5 1.6 1.7 1.8 1.9 2.0 2.1

Cholesterol, mmol/L

Fre

quen

cy o

r P

roba

bilit



>cut-off

ln(cholesterol)


For one sampling

For two samplings


4.0 5.0 6.0 7.0 8.0

0.0

0.2

0.4

0.6

0.8

1.01.3 1.4 1.5 1.6 1.7 1.8 1.9 2.0 2.1

Cholesterol, mmol/L

Fre

quen

cy o

r P

roba

bilit



>cut-off

ln(cholesterol)



4.0 5.0 6.0 7.0 8.0

0.0

0.2

0.4

0.6

0.8

1.01.3 1.4 1.5 1.6 1.7 1.8 1.9 2.0 2.1

Cholesterol, mmol/L

Fre

quen

cy o

r P

roba

bilit



>cut-off

ln(cholesterol)



Cholesterol:


Cumulated frequency (probability functions), one sampling

Now we take a sample within a small interval of healthy set-pointsand multiply with the probability of these set-points exceed the cut-offto get the FP for healthy with this set-point

and by repeating the process for all intervals we get the distribution of origins of set-points of FP

Distribution of Set-Points of Low Risk Individuals Measured as High Risk Above Cut-Off - No Bias

7.06.05.04.0 8.00.0000

0.0005

0.0010

0.0015

0.0020

0.00251.3 1.4 1.5 1.6 1.7 1.8 1.9 2.0 2.1

Cholesterol, mmol/L

Fre

qu

en

cy

Cut-off = 1.8262 = 6.21

CVAnalytical = 0 %

CVAnalytical = 2 %

CVAnalytical = 4 %

CVAnalytical = 6 %

CVAnalytical = 8 %

CVAnalytical = 10 %

Mean = 1.66 = 5

CVBetween = 13.90%

CVWithin = 6.00%

Bias % = 0.0


ln(Cholesterol) mmol/mol


Distribution of Set-Points of Low Risk Individuals Measured as High Risk Above Cut-Off - Bias = + 4 %

8.07.06.04.0 5.0

0.0000

0.0005

0.0010

0.0015

0.0020

0.00251.3 1.4 1.5 1.6 1.7 1.8 1.9 2.0 2.1

Cholesterol, mmol/L

Fre

quen

cy

Apparant Cut-off = 1.786

Assumed Cut-off = 1.826

CVAnalytical = 0.000






'Mean' = 1.66

sBetween = 0.139

sWithin = 0.06

True Bias % = 4


Apparant Cut-off Assumed Cut-offOrigin of Set-Points for

varying imprecision


Cholesterol:

Origin of set-points

Cumulated frequency (probability functions), one sampling

Distribution of Set-Points of Low Risk Individuals Measured as High Risk Above Cut-Off - No Bias, Two samplings

7.06.05.04.0 8.00.0000

0.0005

0.0010

0.0015

0.0020

0.00251.3 1.4 1.5 1.6 1.7 1.8 1.9 2.0 2.1

Cholesterol, mmol/L

Fre

qu

en

cy

Cut-off = 1.8262 = 6.21

CVAnalytical = 0 %

CVAnalytical = 2 %

CVAnalytical = 4 %

CVAnalytical = 6 %

CVAnalytical = 8 %

CVAnalytical = 10 %

Mean = 1.66 = 5

CVBetween = 13.90%

CVWithin = 6.00%

Bias % = 0.0




Distribution of Set-Points of Low Risk Individuals Measured as High Risk, Bias = + 4 %, Two samplings

8.07.06.04.0 5.0

0.0000

0.0005

0.0010

0.0015

0.0020

0.00251.3 1.4 1.5 1.6 1.7 1.8 1.9 2.0 2.1

Cholesterol, mmol/L

Fre

quen

cyApparant Cut-off = 1.786

Assumed Cut-off = 1.826







'Mean' = 1.66

sBetween = 0.139

sWithin = 0.06

True Bias % = 4


Apparant Cut-off Assumed Cut-offOrigin of Set-Points for

varying imprecision

One sampling

Two samplings

Bias = 0 % Bias = +4 %

Apparent cut-off

Assumed cut-off

Distribution of Set-Points of Low Risk Individuals Measured as High Risk Above Cut-Off - No Bias

7.06.05.04.0 8.00.0000

0.0005

0.0010

0.0015

0.0020

0.00251.3 1.4 1.5 1.6 1.7 1.8 1.9 2.0 2.1

Cholesterol, mmol/L

Fre

qu

en

cy

Cut-off = 1.8262 = 6.21

CVAnalytical = 0 %

CVAnalytical = 2 %

CVAnalytical = 4 %

CVAnalytical = 6 %

CVAnalytical = 8 %

CVAnalytical = 10 %

Mean = 1.66 = 5

CVBetween = 13.90%

CVWithin = 6.00%

Bias % = 0.0





Cholesterol:

For one sampling

For two samplings

Percentage of false positive

As function of bias % for varying percentages of imprecision

Inf luence of Bias and Imprecision on Percentage Low Risk

Individuals w ith Cholesterol above 6.2 mmol/L for one Sample

Bias = 0%

0

5

10

15

20

25

-10 -8 -6 -4 -2 0 2 4 6 8 10Bias %

Per

cent

age

FP


Influence of Bias and Imprecision on Percentage of Low Risk Individuals w ith Cholesterol above 6.2 mmol/L, tw o Samples

Bias = 0%

0

5

10

15

20

25

-10 -8 -6 -4 -2 0 2 4 6 8 10Bias %

Per

cent

age

FP



Biological Variables: HbA1c and Cholesterol

Age and Gender

Within- and Between-Subject Biological Variation

Ethnicity, Lifestyle, Environment, Subclinical Diseases, etc.

Seasonal variation

CholesterolHbA1c

Garde et al. Clin Chem 2000;46:551

1. Sharp decision limits are extremely sensitive to analytical bias

2. Use of two measurements in diagnosis reduces the effect of performance errors

3. Separate specifications for analytical bias and imprecision is recommended

4. Biological variation and other information should be considered in decision limits

5. A concentration related probability function as an alternative to sharp decision limit is proposed

Conclusions


00.10.20.30.40.50.60.70.80.9

1

0 20 40 60 80 100 120 140 160 180 200

Decision Value

Pro

bab

ilit

y

A concentration related probability function as an alternative to sharp decision limit


Proposal:

(Arbitrary Units)

Probability of disease

influence of analytical bias and imprecision on guideline-driven medical decision limits per hyltoft...

Documents