![Page 1: Reference Intervals for New Methods Dr Graham Jones Department of Chemical Pathology St Vincent’s Hospital, Sydney](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649d3e5503460f94a166dc/html5/thumbnails/1.jpg)
Reference Intervals for New Methods
Dr Graham Jones
Department of Chemical Pathology
St Vincent’s Hospital, Sydneywww.sydpath.stvincents.com.au
![Page 2: Reference Intervals for New Methods Dr Graham Jones Department of Chemical Pathology St Vincent’s Hospital, Sydney](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649d3e5503460f94a166dc/html5/thumbnails/2.jpg)
Reference Intervals for new Methods
Subtitle: “Reference Intervals”
![Page 3: Reference Intervals for New Methods Dr Graham Jones Department of Chemical Pathology St Vincent’s Hospital, Sydney](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649d3e5503460f94a166dc/html5/thumbnails/3.jpg)
Contents
• Introduction to reference intervals
• Reference intervals for the new method
– Derive de-novo
– Transfer from old method– Literature– Other Laboratories
• Conclusions
![Page 4: Reference Intervals for New Methods Dr Graham Jones Department of Chemical Pathology St Vincent’s Hospital, Sydney](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649d3e5503460f94a166dc/html5/thumbnails/4.jpg)
Defining Reference Intervals
• Central 95% of results from a reference population– IFCC/NCCLS definition
• Excludes 2.5% above and below interval• For healthy population are “Health-associated
Reference intervals”• Can be any population, but must be defined
– eg, pregnant, premature, hospitalised, treated.
![Page 5: Reference Intervals for New Methods Dr Graham Jones Department of Chemical Pathology St Vincent’s Hospital, Sydney](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649d3e5503460f94a166dc/html5/thumbnails/5.jpg)
Other forms:
• Other statistical cuttoffs– Troponin: 99th centile of healthy population– Apo (a): 80th centile of total population
• Recommended interval (decision point)– Impaired fasting glucose (6.1 - 6.9 mmol/L)– Target LDL concentration (<2.0 mmol/L)
• Therapeutic Interval– Drugs, INR, APTT, TSH
![Page 6: Reference Intervals for New Methods Dr Graham Jones Department of Chemical Pathology St Vincent’s Hospital, Sydney](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649d3e5503460f94a166dc/html5/thumbnails/6.jpg)
Current Paradigm
• Based on recommendations from the NCCLS and the IFCC
• Repeated in Product Information from most reagent suppliers
• Encoded in the NATA summary of ISO/IEC guide 17025.– laboratories may perform their own detailed reference
interval studies
or
– may validate reference intervals published elsewhere for their own methods and populations
![Page 7: Reference Intervals for New Methods Dr Graham Jones Department of Chemical Pathology St Vincent’s Hospital, Sydney](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649d3e5503460f94a166dc/html5/thumbnails/7.jpg)
Generating a new reference interval
• Define and select reference population*
• Define collection conditions and numbers
• Collect samples
• Analyse samples
• Perform statistical evaluation*
• Put into practice
• Tietz Textbook covers standard approach very well (HE Solberg)
![Page 8: Reference Intervals for New Methods Dr Graham Jones Department of Chemical Pathology St Vincent’s Hospital, Sydney](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649d3e5503460f94a166dc/html5/thumbnails/8.jpg)
Define Reference Population
• Source – eg blood bank, lab volunteers, students
• Numbers• Exclusions• Likely Partitioning
– Age– Sex– Other
• Difficult to get extremes of age and high numbers
![Page 9: Reference Intervals for New Methods Dr Graham Jones Department of Chemical Pathology St Vincent’s Hospital, Sydney](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649d3e5503460f94a166dc/html5/thumbnails/9.jpg)
Study Imprecision
• Estimates of reference limits has limitations
• Expressed as the confidence interval of the Reference Limits, eg 90% CI of the upper and lower reference limits
• Confidence intervals decrease as the number of people in the study increases.
Large n
Small n
![Page 10: Reference Intervals for New Methods Dr Graham Jones Department of Chemical Pathology St Vincent’s Hospital, Sydney](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649d3e5503460f94a166dc/html5/thumbnails/10.jpg)
Non-parametric statistics
• Lowest number where error envelope can be calculated is 120
• For n=120– 2.5th centile is 4th lowest result– 90% confidence limit for LRL is lowest sample
and 7th lowest sample• These values often very scattered giving wide
intervals
![Page 11: Reference Intervals for New Methods Dr Graham Jones Department of Chemical Pathology St Vincent’s Hospital, Sydney](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649d3e5503460f94a166dc/html5/thumbnails/11.jpg)
1 2 Percent: 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+ 1 | . . . .� 2 | . � � � � � � � � � . . . 3 | . .� � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 4 | >� � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 5 | >� � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 6 | .� � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 7 | . . .� � � � � � � � � � � � � � � � � � � � � � � � � � � � � 8 | . . . .� � � � � � � � � � � � � � 9 | . . . .� � � � � � � � 10 | . . . .� � � 11 | . . . .� � � 12 | . . � � . . 13 | . . . .� 14 | . . . .� 15 | . . . .� 16 | � . . . . 17 | . . . . 18 | . . . . 19 | . . . . +--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+--+
11
19
27
33
41
49
57
NORIP STUDYFemale ALT (n=1220)
Female Upper Reference Limit: 45.6 (90% CI 42.5 – 49.3, n=1220)Male Upper Reference Limit: 68 (90% CI 63.4 – 73.6, n=1080)
ALT
(U
/L)
![Page 12: Reference Intervals for New Methods Dr Graham Jones Department of Chemical Pathology St Vincent’s Hospital, Sydney](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649d3e5503460f94a166dc/html5/thumbnails/12.jpg)
Generating Intervals
• Is hard to do well
• Requires time and effort and money
• But any local data may be very useful for validation of other intervals
![Page 13: Reference Intervals for New Methods Dr Graham Jones Department of Chemical Pathology St Vincent’s Hospital, Sydney](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649d3e5503460f94a166dc/html5/thumbnails/13.jpg)
“Impossible” Intervals
• Some reference intervals are essentially impossible to produce from local studies:– Paediatric intervals– Stages of pregnancy (eg hCG in 5th week)– Stages of menstrual cycle– Nutritional parameters
• Reflects local diet• May normalise deficiency state
![Page 14: Reference Intervals for New Methods Dr Graham Jones Department of Chemical Pathology St Vincent’s Hospital, Sydney](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649d3e5503460f94a166dc/html5/thumbnails/14.jpg)
Transfer Intervals from previous method
• Implies previous intervals are good– Check source and validity
• Transfer requires good correlation
• Advantage is clinical acceptance
– Note: much the following data related to introduction of a Bayer Centaur for Vitamin B12.
![Page 15: Reference Intervals for New Methods Dr Graham Jones Department of Chemical Pathology St Vincent’s Hospital, Sydney](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649d3e5503460f94a166dc/html5/thumbnails/15.jpg)
Transferring Intervals
Wide range of results, assayed over several days, excellent correlationAnd linearity. Transfer with no problem
Total Protein
y = 1.0007x - 0.5037
R2 = 0.9957
0
20
40
60
80
100
120
140
0 20 40 60 80 100 120 140
Modular <P>
AU
2700
![Page 16: Reference Intervals for New Methods Dr Graham Jones Department of Chemical Pathology St Vincent’s Hospital, Sydney](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649d3e5503460f94a166dc/html5/thumbnails/16.jpg)
Vitamin B12 y = 0.82x + 38
0
50
100
150
200
250
300
350
400
450
500
0 100 200 300 400 500
Access
Ce
nta
ur
BECKMAN107 (bottom of
normal)
133 (top of deficient)
BAYER181 (top of deficient)
156 (bottom of normal)
126SydPath
95% Confidence LimitsSlope: 0.78 – 0.86Intercept: 28 - 48
Transferring Intervals – more difficult
![Page 17: Reference Intervals for New Methods Dr Graham Jones Department of Chemical Pathology St Vincent’s Hospital, Sydney](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649d3e5503460f94a166dc/html5/thumbnails/17.jpg)
Correlation Data
• Patient samples
• Focus on results near limits
• Beware effect of extreme values on statistics– Passing and Bablock preferred to linear
regression
• Use correlation data from several days and calibrations
• Review source of previous Intervals
![Page 18: Reference Intervals for New Methods Dr Graham Jones Department of Chemical Pathology St Vincent’s Hospital, Sydney](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649d3e5503460f94a166dc/html5/thumbnails/18.jpg)
Validation of reference intervals
• NCCLS protocol
• Measure 20 samples appropriate for reference interval on new method
• Exclude outliers
• If 2 or fewer are outside proposed inetrvals– Accept intervals
• If >2 are outside proposed intervals– Measure another 20– If 2 or fewer are outside – accept intervals
• Cannot detect overly wide intervals
![Page 19: Reference Intervals for New Methods Dr Graham Jones Department of Chemical Pathology St Vincent’s Hospital, Sydney](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649d3e5503460f94a166dc/html5/thumbnails/19.jpg)
Review Previous Method
• Previous method may have significant amounts of data (information)
• For many assays many of the results will be on “normal” patients
• For all assays will allow assessment of previous reference intervals
• Methods:– Inspection– Frequency histograms (all data, some data)– Formal methods (Bhattacharya)
![Page 20: Reference Intervals for New Methods Dr Graham Jones Department of Chemical Pathology St Vincent’s Hospital, Sydney](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649d3e5503460f94a166dc/html5/thumbnails/20.jpg)
Access126 pmol/L5.8% rate of “low” results
Centaur - predicted180 pmol/L cuttoff16% positive rate
0
100
200
300
400
500
60 120
180
240
300
360
420
480
540
600
660
720
780
840
900
960
1020
Centaur B12 (predicted)
050
100150200250300350
6 66 126
186
246
306
366
426
486
546
606
666
726
786
846
906
966
1026
1086
1146
1206
Access B12 (actual)
Assess effect of possibleAssay change
![Page 21: Reference Intervals for New Methods Dr Graham Jones Department of Chemical Pathology St Vincent’s Hospital, Sydney](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649d3e5503460f94a166dc/html5/thumbnails/21.jpg)
Data Mining old results
• Bhattacharya, LG. Journal of the Biometric Society. 1967;23:115-135.
• Example data: Frequency Distribution of the forkal length of the Porgy caught by pair-trawl fishery in the East China Sea.
![Page 22: Reference Intervals for New Methods Dr Graham Jones Department of Chemical Pathology St Vincent’s Hospital, Sydney](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649d3e5503460f94a166dc/html5/thumbnails/22.jpg)
Bhattacharya
• Assumes Gaussian (or Log Gaussian) distributions
• Assumes a significant proportion of requests are on unaffected individuals
0500
10001500200025003000350040004500
0 0.05 0.1 0.15 0.2
patient values
Bhattacharya
Creatinine
![Page 23: Reference Intervals for New Methods Dr Graham Jones Department of Chemical Pathology St Vincent’s Hospital, Sydney](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649d3e5503460f94a166dc/html5/thumbnails/23.jpg)
Data Mining
• Bhattacharya ignores effects of outliers and samples not part of majority distribution.
• Reference intervals based on majority.
0500
10001500200025003000350040004500
0 0.05 0.1 0.15 0.2
patient values
Bhattacharya
Creatinine (mmol/L)
![Page 24: Reference Intervals for New Methods Dr Graham Jones Department of Chemical Pathology St Vincent’s Hospital, Sydney](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649d3e5503460f94a166dc/html5/thumbnails/24.jpg)
Literature
• Look for same method
• Equivalent population
• Sources– Peer-reviewed publications– Gray Literature
• Abstracts (eg AACB, AACC, ACB)
– Company literature• Product information (PI)
• Other
![Page 25: Reference Intervals for New Methods Dr Graham Jones Department of Chemical Pathology St Vincent’s Hospital, Sydney](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649d3e5503460f94a166dc/html5/thumbnails/25.jpg)
Literature sources
• Vital where population reference intervals may be of limited use
• Dietary factors
• Special groups– Eg paediatrics
• Numbers are prohibitive– eg 99th centile for troponins
• Following examples taken from SydPath data for creatinine (Roche) and Vitamin B12 (Centaur)
![Page 26: Reference Intervals for New Methods Dr Graham Jones Department of Chemical Pathology St Vincent’s Hospital, Sydney](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649d3e5503460f94a166dc/html5/thumbnails/26.jpg)
Combining data
• Local and blood bank (M 101, F 110) M: 62 – 105 umol/L F: 51 – 82 umol/L.• Literature: • South Australia (M 293, F 269) Mazzachi BC et al,
Clin Lab. 2000;46:53-55 M: 62 – 106 umol/L F: 44 – 80 umol/L• Germany (M 127; F125) Junge et al. Clin Chem
Acta. 2004;344:137-148 M: 63 – 103 umol/L F: 48 – 85 umol/L• Values rounded out as follows: M: 60 – 110 mmol/L F: 40 – 90 mmol/L
![Page 27: Reference Intervals for New Methods Dr Graham Jones Department of Chemical Pathology St Vincent’s Hospital, Sydney](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649d3e5503460f94a166dc/html5/thumbnails/27.jpg)
150 300 450 600 750
Vitamin B12 – ACS:180 Klee 2000 (pmol/L)
180 pmol/L
Literature Sources - distribution
![Page 28: Reference Intervals for New Methods Dr Graham Jones Department of Chemical Pathology St Vincent’s Hospital, Sydney](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649d3e5503460f94a166dc/html5/thumbnails/28.jpg)
Homocysteine and Methylmalonic acid relative to serum B12 (Centaur)
Homocysteine
MMA
1562 people, age >65. MMA and Homocysteine.B12 measured on Bayer CentaurBin width 50 pmol/L. Red Arrow 200 pmol/L.
- Clarke et al, Am J Clin Nutrit. 2003;77:1241-7.
![Page 29: Reference Intervals for New Methods Dr Graham Jones Department of Chemical Pathology St Vincent’s Hospital, Sydney](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649d3e5503460f94a166dc/html5/thumbnails/29.jpg)
Dorevich PathologySikaris et al25,201 B12 measurementsACS:180 and Bayer CentaurCentral 95% of results with Normal Hb and MCV: 178 - 741 pmol/L
![Page 30: Reference Intervals for New Methods Dr Graham Jones Department of Chemical Pathology St Vincent’s Hospital, Sydney](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649d3e5503460f94a166dc/html5/thumbnails/30.jpg)
Vitamin B12 v MCV
• SydPath Data (3 months, 1497 results)
• Beckman-Coulter Access
020406080
100120140160
0 500 1000 1500
Vitamin B12 (Access, pmol/L)
MC
V
126
![Page 31: Reference Intervals for New Methods Dr Graham Jones Department of Chemical Pathology St Vincent’s Hospital, Sydney](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649d3e5503460f94a166dc/html5/thumbnails/31.jpg)
147
441
294
VB
12, p
mol
/L
588
140 pmol/L
181 pmol/L
156 pmol/L
Product Information
![Page 32: Reference Intervals for New Methods Dr Graham Jones Department of Chemical Pathology St Vincent’s Hospital, Sydney](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649d3e5503460f94a166dc/html5/thumbnails/32.jpg)
Centaur Vitamin B12
• 6 studies using Centaur or ACS:180
• Product information• 3 x refereed publications• 1 x AACB abstract• 1 x local study (NZ)
• Data combined to make reference interval– Deficient <120 pmol/L– Indeterminate 120 – 180 pmol/L– Replete >180 pmol/L
![Page 33: Reference Intervals for New Methods Dr Graham Jones Department of Chemical Pathology St Vincent’s Hospital, Sydney](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649d3e5503460f94a166dc/html5/thumbnails/33.jpg)
Manufacturer’s Interval: well-defined population, appropriate exclusionsBut: Outliers?, bi/trimodal distribution?
Product Information…
0
5
10
15
20
25
30
0 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40
# o
f R
esu
lts
Recommended Interval: 3 - 25 (2.5th to 97.5th Centile)90% CI of URL: 19 - 39
![Page 34: Reference Intervals for New Methods Dr Graham Jones Department of Chemical Pathology St Vincent’s Hospital, Sydney](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649d3e5503460f94a166dc/html5/thumbnails/34.jpg)
Other laboratories
• If someone has done the work, and uses the same method, review their work and apply the intervals.
• Need to verify assay bias.
• Collaborative effort between several labs with the same method may be a powerful method of setting reference intervals– Spanish Group– NORIP: http://wip.furst.no/norip/
![Page 35: Reference Intervals for New Methods Dr Graham Jones Department of Chemical Pathology St Vincent’s Hospital, Sydney](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649d3e5503460f94a166dc/html5/thumbnails/35.jpg)
Combining Laboratories
• 13 Spanish laboratories (all Centaurs)• 11 – 15 samples from each laboratory (tot 150 samples)• Combined data used for Reference Intervals
– Ferre-Masferrer et al. Clin Chem Lab Med 2001;39:166-169
![Page 36: Reference Intervals for New Methods Dr Graham Jones Department of Chemical Pathology St Vincent’s Hospital, Sydney](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649d3e5503460f94a166dc/html5/thumbnails/36.jpg)
Other Sources: accuracy base
• In order to share method-specific literature need to ensure assay accuracy.
• “Is my Bloggs method for X working the same as everyone else’s Blogg’s method?”
• QAP results– measure QAP samples– look up results for method group
• QC material target values
• Shared samples
![Page 37: Reference Intervals for New Methods Dr Graham Jones Department of Chemical Pathology St Vincent’s Hospital, Sydney](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649d3e5503460f94a166dc/html5/thumbnails/37.jpg)
Comparison with QAP targets
QAP Endocrine program - free T3
y = 0.9382x + 0.4922
R2 = 0.9944
0
2
4
6
8
10
12
14
0 2 4 6 8 10 12 14
QAP Centaur Median
Syd
Pat
h R
esu
lts
Method-specific medians (and scatter) available on QAP website
![Page 38: Reference Intervals for New Methods Dr Graham Jones Department of Chemical Pathology St Vincent’s Hospital, Sydney](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649d3e5503460f94a166dc/html5/thumbnails/38.jpg)
Plea to manufacturers...
• Searching refereed literature by trade names can be difficult– ie Abbott, Elecsys, Immulite, Vitros are terms
that are not often searchable in Medline, pubmed etc
• If companies keep a resource library of information it would be very useful.
• Note “google Scholar” can be useful– http://scholar.google.com
![Page 39: Reference Intervals for New Methods Dr Graham Jones Department of Chemical Pathology St Vincent’s Hospital, Sydney](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649d3e5503460f94a166dc/html5/thumbnails/39.jpg)
Clinical Input
• Previous slides about Vitamin B12 are taken from a presentation to haematologists at St Vincent’s Hospital
• Actively seeking their input on decision points
• Allows inclusiveness and practical input
![Page 40: Reference Intervals for New Methods Dr Graham Jones Department of Chemical Pathology St Vincent’s Hospital, Sydney](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649d3e5503460f94a166dc/html5/thumbnails/40.jpg)
Putting it all together
• Different sources will give (slightly) different values.
• Judgement is required to combine data
• Other factors include:– Precision of intervals– Long term precision of assay(s)– Biological variation– Rounding for ease of memory– Partitioning
![Page 41: Reference Intervals for New Methods Dr Graham Jones Department of Chemical Pathology St Vincent’s Hospital, Sydney](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649d3e5503460f94a166dc/html5/thumbnails/41.jpg)
Implementing
• Recommend temporary footnote
• eg change in method and change intervals, see lab for further details
• Make further details available if needed– source document (NATA)– Handout– Website
![Page 42: Reference Intervals for New Methods Dr Graham Jones Department of Chemical Pathology St Vincent’s Hospital, Sydney](https://reader035.vdocuments.mx/reader035/viewer/2022062516/56649d3e5503460f94a166dc/html5/thumbnails/42.jpg)
Conclusions
• A new method is a good time to review reference intervals
• Uncritical transfer of old intervals is bad practice
• Many sources of information can be used
• Judgement is required for final decision
• Working with other labs may be of great benefit