Metrology in medicine

Martin Turner Senior Research Associate &Douglas Joseph Fellow,Dept of Anaesthetics, University of Sydney

Peter C Kam Prof & Head of Department,Dept of Anaesthetics, UNSW.

A Barry Baker Prof & Head of Department,Dept of Anaesthetics, University of Sydney.

Acknowledgements: RB Frenkel, NML

AJ Coleman, Guy’s & St Thomas’Hospital, UK

Measurements in medicine:

Aid diagnosis

• absolute values

Guide treatment:

• absolute values

• relative values or changes/trends(e.g. peak flow in asthma)

Old instruments vs modernEarly medical

instruments:

• Simple, mechanical

• Faults were self-evident

Modern instruments:• Black (coloured) box

• Software-based

• Digital display

• Faults & lack of calibrationnot readily evident

Characteristics of medical measurements

High variability(both intra– and inter–individual)

Often indirect(e.g. cardiac output)

Results often depend strongly on measurement technique

Often uncertainty 10 – 20% is acceptable

RiskMedical measurements are made on

sick people:

• most diagnostic & therapeutic procedures have associated risks

• a degree of risk associated with measurements is acceptable

Risk and variability

Clinicians are familiar with the combination of variability & risk

Many clinicians:• assume variability due to lack of calibration is

swamped by biological variability• do not see a need for traceable calibration of

medical instruments

Many medical instruments are not traceably calibrated

Diagnoses rarely depend on a single measurement

Good clinicians:

• know what results to expect

• disregard unusual measurements in the face of conflicting clinical evidence

Mitigating factors

Laws, regulationsstandards & accreditation

Therapeutic Goods Authority

Falls under the Federal Dept of Health and Aging

New medical devices must be licensed by the TGA

Test equipment used to assess new equipment is required to be traceably calibrated

Maintenance of existing medical equipment?

StandardsAS/NZS 3551:1996. Technical

management programs for medical devices.

• TEST EQUIPMENT: Test equipment used shall be regularly calibrated … to achieve traceability of measurement.

• AS/NZS 3551 is not enforced

• AS/NZS 3551 is ignored in some hospitals

Accreditation The Australian Council on Healthcare

Standards (ACHS) accredits Hospitals• metrology issues are not prominent

NATA accredits Pathology & Biochem labs (Medical Testing Labs)

Some professional bodies accredit their labs & practitioners

Quality control of other measurements appears to be at the discretion of individual clinicians, groups or hospitals

Evidence-based medicine

EBM is

• The conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patientsBMJ 1996 312(7023):71-2.

Presently there is no evidence that uncalibrated medical instruments cause adverse outcomes (Wilson et al. Med J Aust 1995;163:458. Med J Aust 1999; 170:411)

Is traceable calibration likely to improve medical outcomes?

Case study: Blood pressure

Most commonly measured physiological variable

Diagnosis of hypertension depends entirely on BP measurements

29% of Australians 25 yrs have hypertension (NHF)

Hypertension is an independent risk factor for coronary heart disease and cardiovascular disease.

Cardiovascular diseases caused 40% of all deaths in Australia in 1998 (NHF).

What is the effect of measurement error on the diagnosis of hypertension?

What is the range of measurement error?

What is the sensitivity to those errors?

AS EN 1060.1 2002 Non-invasive sphygmomanometers – General requirements requires |error| < 3 mm Hg

UK Sphyg survey (J Hum Hyperten 15:587 2001)

-30 -20 -10 0 10 20 300

20

40

60

80

100

BP error (mm Hg)

cu

mu

lati

ve

dis

t (%

)

949 Hg sphygs

513 aneroid sphygs

|error| > 3 mm Hg: 16%

|error| > 5 mm Hg: 9.3%

|error| > 10 mm Hg: 3.7%

1 in 54 UK GPpractices had sphygscalibrated regularly

Distribution of diastolic BP

66

16

105

2 10

10

20

30

40

50

60

70

< 80 80-84 85-89 90-94 95-99 > 100

Diastolic BP (mm Hg)

% o

f s

ub

jec

ts

Source: Joffres MR et al. Can Med Assoc J 146(11):1997 - 2005 1992Data acquired 1986-1990; N = 20582; data include treated subjects (10% of total).

80 100 120 140 16010

0

101

102

Blood pressure (mm Hg)

% s

ubje

cts

with

pre

ssur

e >

xdiastolicsystolic

cbxaxxp 2

10)(

Prob. that patient has true BP > x: p(x)

Classification threshold: xo

Measurement error: x

Probability of classifying a patient hypertensive: p(xo–Δx)

Change in hypertensives: )(

)(

0xp

xxpf o

-10 -8 -6 -4 -2 0-90

-80

-70

-60

-50

-40

-30

-20

-10

0

measurement error (mm Hg)

hyp

ert

ensi

ve p

atie

nts

mis

sed

(%

)

Systolic

Diastolic85 mm Hgthreshold

90 mm Hg

95 mm Hg

0 2 4 6 8 100

100

200

300

400

500

600

measurement error (mm Hg)

addi

tiona

l hyp

erte

nsiv

e p

atie

nts

(%)

Systolic

Diastolic

85 mm Hg

90 mm Hg

95 mm Hgthreshold

Effects of systematic error in BPat xo = 95 mm Hg

• +3 mm Hg: of 9 patients called hypertensive 4 are not

• –3 mm Hg: half of true hypertensives missed

• +5 mm Hg: of 13 patients called hypertensive 8 are not

• –5 mm Hg: 2/3 of hypertensives missed

Determine the performance of sphygs in Australian GP practices

Analyse the distribution of BP in Australia

Estimate:• the number of normotensive people treated for

hypertension• the number of untreated hypertensives• the number of cardiovascular deaths due to

inadequate sphygmomanometer calibration

Further work

Other measurements

Temperature:• IR thermometers:

– bias up to 2.3°C– 1.5°C scatter

Spirometry:• Electronic spirometers

– Type tested (inadequately?)– In use calibration is inadequate

These examples may represent the tip of the medical metrology iceberg in Australia

Inadequately calibrated medical instruments are an unrecognised cause of preventable medical errors

Traceable calibration of medical measurement systems would:

• Improve the quality of healthcare

• Reduce long term healthcare costs

Conclusions

End

Extras

Temperature

Tympanic IR thermometers are replacing Hg-in-glass thermometers

Imamura et al. (Tokyo 1998) compared four different IR thermometers with tympanic thermocouples in vivo.

The

rmoc

oupl

e –

IR (

ºC)

Imamura et al. Acta Anaesthesiol Scand 42:1222-6 1998.

The

rmoc

oupl

e –

IR (

ºC)

Imamura et al. Acta Anaesthesiol Scand 42:1222-6 1998.

Spirometry

Commonly used for diagnosis & treatment of asthma and COPD

2.2 million asthmatics in Australia

397 asthma deaths in Australia in 2002

0 0.5 1 1.5 2

0

2

4

6

8

10

12

time (s)

flo

w (

L/s

)

ATS standard curves

• Forced vital capacity (FVC)• FEV1

• Peak flow

26

1

7

Hankinson & Crapo, Am J Respir Crit Care Med 1995; 152:696

Spirometer calibration

• At manufacture: Type-tested against 26 ATS curves using servo-driven syringe

• In use:

– volume checks only

– hand-driven 3 litre syringe

• Servo-driven syringe has no correction for gas compression

• Many hand-driven syringes are not regularly calibrated

• Forced expiratory flow has high harmonic content, so static calibration (0 Hz only) is completely inadequate.

The

rmoc

oupl

e –

IR (

ºC)

Imamura et al. Acta Anaesthesiol Scand 42:1222-6 1998.

Systematicerror

(mm Hg)

Diastolic Systolic*

85† 90 95 –

–5 –57(1)‡ –62(1) –67(1) –30(2)

–3 –39(1) –44(1) –48(1) –19(2)

–1 –15(0·4) –17(0·2) –19(0·5) –7(1)

+1 16(1) 20(0.3) 23(1) 7(1)

+3 55(3) 68(1) 83(2) 24(3)

+5 102(7) 132(4) 166(5) 43(5)

• Professional bodies:

– accredit some labs(Thoracic Society of Australia & New Zealand accredits lung function labs)

– certify technicians in some fields(Australian & NZ Soc of Respiratory Science certifies Respiratory Function Scientists)

– produce guidelines

BP measurement error

• 2001: UK GP sphygmomanometers:

3.7% had |error| > 10 mm Hg

9.3% had |error| > 5 mm Hg

16% had |error| > 3 mm Hg

1 in 54 UK GP practices had sphygs regularly calibrated

• 1995/9: Australian sphygs (Newcastle):

Sphyg maintenance: ‘poor’J Qual Clin Pract. 1995;15:17-22, 1999;19:95-8