evaluation of a rapid lateral flow calprotectin test for ... · 63 intraoperative findings with...

17
Title: Evaluation of a rapid lateral flow calprotectin test for the diagnosis of prosthetic joint 1 infection. 2 Authors: Alexander J. Trotter 1,2 , Rachael Dean 1,2 , Celia E. Whitehouse 3 , Jarle Mikalsen 4 , Claire 3 Hill 2,3 , Roxanne Brunton-Sim 3 , Gemma L. Kay 2 , Majeed Shakokhani 3 , Alexander Durst 3 , John 4 Wain 1,2 , Iain McNamara 1,3, *, Justin O’Grady 1,2, * 5 6 Affiliations: 1 University of East Anglia, Norwich, UK. 2 Quadram Institute Bioscience, Norwich, UK. 7 3 Norfolk and Norwich University Hospitals Foundation Trust, Norwich, UK. 4 Lyfstone, Tromsø, 8 Norway. 9 10 Corresponding Author: Alexander J. Trotter. Quadram Institute Bioscience, Norwich Research Park, 11 Norwich, UK, NR4 7UQ. [email protected]. 12 13 *authors contributed equally 14 15 Running title: Evaluation of a lateral flow test for PJI diagnosis 16 17 Keywords: Prosthetic Joint Infection, PJI, calprotectin, synovial fluid, rapid diagnostics, inflammation 18 biomarker 19 20 21 22 23 . CC-BY-NC 4.0 International license It is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. was not certified by peer review) (which The copyright holder for this preprint this version posted August 17, 2019. . https://doi.org/10.1101/19004473 doi: medRxiv preprint

Upload: others

Post on 26-Jul-2020

3 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Evaluation of a rapid lateral flow calprotectin test for ... · 63 intraoperative findings with selected inflammatory markers and culture results. MSIS criteria are 64 useful but

Title: Evaluation of a rapid lateral flow calprotectin test for the diagnosis of prosthetic joint 1

infection. 2

Authors: Alexander J. Trotter1,2, Rachael Dean1,2, Celia E. Whitehouse3, Jarle Mikalsen4, Claire 3

Hill2,3, Roxanne Brunton-Sim3, Gemma L. Kay2, Majeed Shakokhani3, Alexander Durst3, John 4

Wain1,2, Iain McNamara1,3,*, Justin O’Grady1,2,* 5

6

Affiliations: 1University of East Anglia, Norwich, UK. 2Quadram Institute Bioscience, Norwich, UK. 7

3Norfolk and Norwich University Hospitals Foundation Trust, Norwich, UK. 4Lyfstone, Tromsø, 8

Norway. 9

10

Corresponding Author: Alexander J. Trotter. Quadram Institute Bioscience, Norwich Research Park, 11

Norwich, UK, NR4 7UQ. [email protected]. 12

13

*authors contributed equally 14

15

Running title: Evaluation of a lateral flow test for PJI diagnosis 16

17

Keywords: Prosthetic Joint Infection, PJI, calprotectin, synovial fluid, rapid diagnostics, inflammation 18

biomarker 19

20

21

22

23

. CC-BY-NC 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. was not certified by peer review)

(whichThe copyright holder for this preprint this version posted August 17, 2019. .https://doi.org/10.1101/19004473doi: medRxiv preprint

Page 2: Evaluation of a rapid lateral flow calprotectin test for ... · 63 intraoperative findings with selected inflammatory markers and culture results. MSIS criteria are 64 useful but

Abstract 24

Background: Microbiological diagnosis of prosthetic joint infection (PJI) relies on culture techniques 25

that are slow and insensitive. Rapid tests are urgently required to improve patient management. 26

Calprotectin is a neutrophil biomarker of inflammation that has been demonstrated to be effective for 27

the diagnosis of PJI. A calprotectin based lateral flow test has been developed for the rapid detection 28

of PJI using synovial fluid samples. 29

Methods: A convenience series of 69 synovial fluid samples from patients at the Norfolk and Norwich 30

University Hospitals (NNUH) were collected intraoperatively from 52 hip and 17 knee revision 31

operations. Calprotectin levels were measured using a new commercially available lateral flow assay 32

for PJI diagnosis (Lyfstone). For all samples, synovial fluid was pipetted onto the lateral flow device 33

and the signal was read using a mobile phone app after 15 minutes incubation at room temperature. 34

Results: According to the Musculoskeletal Infection Society (MSIS) criteria, 24 patients were defined 35

as PJI positive and the remaining 45 were negative. The overall accuracy of the lateral flow test 36

against the MSIS criteria was 75%. The test had a sensitivity and specificity of 75% and 76% 37

respectively with a positive predictive value (PPV) of 62% and a negative predictive value (NPV) of 38

85%. Discordant results were then reviewed by the clinical team using available patient data to 39

develop an alternative gold standard for defining presence/absence of infection (MSIS+). Compared 40

to MSIS+, the test showed an overall accuracy of 83%, sensitivity and specificity of 95% and 78% 41

respectively, a PPV of 62% and an NPV of 98%. Test accuracy for hip revisions was 77% and for 42

knee revisions was 100%. 43

Conclusions: This study demonstrates that the calprotectin lateral flow assay is an effective diagnostic 44

test for PJI. Our data suggests that the test is likely to generate false positive results in patients with 45

metallosis and gross osteolysis. 46

47

Introduction 48

. CC-BY-NC 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. was not certified by peer review)

(whichThe copyright holder for this preprint this version posted August 17, 2019. .https://doi.org/10.1101/19004473doi: medRxiv preprint

Page 3: Evaluation of a rapid lateral flow calprotectin test for ... · 63 intraoperative findings with selected inflammatory markers and culture results. MSIS criteria are 64 useful but

Each year in the UK, approximately 160,000 primary hip and knee replacements are performed plus 49

an additional 14,000 revision operations. Prosthetic Joint Infection (PJI) is responsible for 14% of 50

these revisions [1]. 51

Diagnosis of PJI remains a challenge and guidelines vary between countries. Low-grade PJIs are 52

particularly difficult to diagnose. They are commonly caused by bacteria which do not have a clear 53

pathogenic role and often contaminate tissue samples, making culture results difficult to interpret [2]. 54

These organisms are also less likely to trigger an increase in inflammatory makers [3]. The inability to 55

differentiate between low-grade infections and aseptic loosening, leads to patients undergoing 56

numerous investigations and unnecessary two-stage revisions. This treatment option comes with a 57

higher cost both to the to the healthcare system and accompanying patient morbidity, along with an 58

associated higher complication rate than one-stage revisions [4]. For this group of patients especially 59

there is a need for a diagnostic test that can reliably exclude PJI. 60

The Musculoskeletal Infection Society (MSIS) published a list of criteria designed to standardise the 61

diagnosis of PJI [5]. The criteria require a diagnostic work-up combining preoperative and 62

intraoperative findings with selected inflammatory markers and culture results. MSIS criteria are 63

useful but fallible, due to the heavy weighting placed on microbiological culture results, which are 64

known to have sub-optimal specificity and sensitivity. 65

In 2018, the MSIS criteria were revised due to the emergence of new PJI biomarkers including serum 66

D-dimer, synovial alpha-defensins and synovial leukocyte esterase [6]. These tests can be expensive 67

and their application in clinical practice is currently limited. An alternative biomarker is synovial fluid 68

calprotectin. Calprotectin is a protein complex released during inflammation that makes up 60% of all 69

soluble proteins in neutrophils [7]. Neutrophils are recruited to sites of inflammation and infection 70

response; therefore, high amounts of neutrophils biomarkers are expected to be seen in infected 71

patient samples [8]. 72

Calprotectin is routinely used to screen for inflammatory bowel disease and has been shown to detect 73

relapses in rheumatoid arthritis [9, 10]. In a recent study, a stool calprotectin test was used off label 74

. CC-BY-NC 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. was not certified by peer review)

(whichThe copyright holder for this preprint this version posted August 17, 2019. .https://doi.org/10.1101/19004473doi: medRxiv preprint

Page 4: Evaluation of a rapid lateral flow calprotectin test for ... · 63 intraoperative findings with selected inflammatory markers and culture results. MSIS criteria are 64 useful but

for PJI diagnosis demonstrating a negative predictive value (NPV) of 94.4% [11]. This demonstrates 75

the utility of calprotectin for PJI diagnosis and the need for a validated lateral flow test. 76

Lyfstone have recently developed a lateral flow calprotectin test for the diagnosis of PJI which has 77

passed the European IVD regulatory approval process (98/79/EC). We conducted the first clinical 78

evaluation of this new technology in a retrospective study on samples from suspected PJI or aseptic 79

loosening cases to assess the sensitivity, specificity, NPV and PPV. 80

81

Materials and Methods 82

A total of 73 synovial fluid samples collected as a convenience series and stored in the Biorepository 83

at the University of East Anglia between February 2016 and January 2019 as part of routine clinical 84

practice from patients treated at the Norfolk and Norwich University Hospitals (NNUH) were 85

retrospectively tested for calprotectin levels with the Lyfstone lateral flow test. Informed consent was 86

obtained from all patients included in the study. Ethical approval was provided by University of East 87

Anglia, Faculty of Medicine and Health Sciences Research Ethics Committee, project reference: 88

2016/17 21 SE. Samples were only tested if of sufficient volume (≥100µL) and results for 89

microbiological culture, frozen-section histology and serum C-Reactive Protein (CRP) were available. 90

Four samples were removed from the analysis due to incomplete/erroneous patient data (two had no 91

pre-operative notes and two were from a patient who had two separate surgeries, but the samples had 92

the same date). The final total of 69 samples were included in the dataset (37 male, 32 female, age 93

range 45-89), 52 were taken during hip revision surgery and 17 from knee revisions. 94

Cases were classified as either infected or aseptic based on the MSIS criteria (Table 1). A case was 95

deemed as infected if the patient presented either with a major criterion, i.e. two or more cultures of 96

the same organism or the presence of a sinus tract; or met three minor criteria i.e. elevated serum CRP 97

levels, purulence, one positive microbiological culture and/or positive histological analysis of 98

periprosthetic tissue for inflammation. Other minor criteria from the 2018 MSIS criteria were not 99

. CC-BY-NC 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. was not certified by peer review)

(whichThe copyright holder for this preprint this version posted August 17, 2019. .https://doi.org/10.1101/19004473doi: medRxiv preprint

Page 5: Evaluation of a rapid lateral flow calprotectin test for ... · 63 intraoperative findings with selected inflammatory markers and culture results. MSIS criteria are 64 useful but

considered as many of those tests are not used as standard in the UK or routine at NNUH. Clinical 100

information was not made available to performers of the index test before testing. 101

102

Major Criteria (one or more of the following) Decision

Two positive cultures of the same organism

Infected Sinus tract with evidence of communication to the joint or visualization of the

prosthesis

Minor Criteria Score Decision

Elevated serum CRP (≥10 mg/L) 2 ≥6 Infected

Positive histology 3

Positive purulence 3 <6 Aseptic

Single positive culture 2

Table 1. Scoring based definition for PJI using available tests from the 2018 MSIS criteria. 103

104

Discrepant results were investigated by one of the senior authors (IM - Consultant Orthopaedic 105

Surgeon) and team by reviewing patient medical records, surgical notes and X-rays to make a definite 106

decision on whether the patients were managed by the clinical team as infected (MSIS+). 107

The Lyfstone calprotectin test was carried out according to the manufacturer’s instructions. Synovial 108

fluid samples (20l) were diluted in a pre-mixed dilution buffer (2 ml – 101x dilution) and added to 109

the test cassette (80l). Gold-conjugated antibody complexes then bind calprotectin and travel along 110

the membrane within the cassette and further bind to immobilised calprotectin-specific antibodies to 111

form a visible test line. Any remaining gold-conjugated antibody not bound to calprotectin is 112

immobilised on a control line. After 15 minutes incubation at room temperature the cassette was 113

photo-imaged, and the calprotectin level calculated by the Lyfstone smartphone application (Figure 114

1). The colour intensity of the test line is proportional to the concentration of calprotectin in the 115

sample. A calprotectin result of ≤14 mg/L was considered a negative result according to the cut-off 116

between low and moderate risk of infection by the Lyfstone smartphone app at the time of testing. 117

. CC-BY-NC 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. was not certified by peer review)

(whichThe copyright holder for this preprint this version posted August 17, 2019. .https://doi.org/10.1101/19004473doi: medRxiv preprint

Page 6: Evaluation of a rapid lateral flow calprotectin test for ... · 63 intraoperative findings with selected inflammatory markers and culture results. MSIS criteria are 64 useful but

Moderate (14-50 mg/L) and severe (>50 mg/L) risk of infection categories were grouped together as 118

positive [12]. 119

120 Figure 1. Lyfstone App report (right) from Lyfstone Calprotectin test (left). 121

122

Microbiological culture was performed on all tissue and fluid specimens (recommended that 3-5 123

tissue/fluid samples sent for testing) for 48 hours on blood agar and chocolate agar, five days on 124

fastidious anaerobe agar, and five days in cooked meat broth before 48-hour subculture on fastidious 125

anaerobe agar, chocolate agar, sabouraud agar and in fastidious antimicrobial neutralization bottles 126

according to UK Standards for Microbiology Investigations B 44 [13]. Histology was performed by 127

frozen section microscopy according to UKAS ISO 8405 [14]; Serum CRP was performed according 128

to UKAS ISO 10294 [15]. 129

130

. CC-BY-NC 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. was not certified by peer review)

(whichThe copyright holder for this preprint this version posted August 17, 2019. .https://doi.org/10.1101/19004473doi: medRxiv preprint

Page 7: Evaluation of a rapid lateral flow calprotectin test for ... · 63 intraoperative findings with selected inflammatory markers and culture results. MSIS criteria are 64 useful but

Results 131

Lyfstone test performance compared to MSIS gold standard 132

The overall sensitivity and specificity of the calprotectin test for the diagnosis of PJI compared to 133

MSIS gold standard was 75% and 75.6% respectively. The PPV was 62.1%, the NPV 85% and the 134

accuracy was 75.4% (Table 2 and 3). 135

Sample

No.

Hip

or

Knee

Histology Culture CRP Sinus

tract Purulence MSIS Calprotectin

1 K negative negative 13 N N negative negative

2 H negative negative 40 N N negative negative

3 H negative negative n/a N N negative negative

4 H negative negative 2 N N negative positive

5 H positive negative 47 Y Y positive positive

6 H negative negative 2 N N negative negative

7 H negative negative 6 N N negative negative

8 K negative negative <1 N N negative negative

9 H negative negative 7 N N negative negative

10 H negative negative 2 N N negative negative

11 H negative negative 7 N N negative positive

12 H negative negative 21 N N negative negative

13 H positive negative 35 Y Y positive positive

14 H negative negative <1 N N negative positive

15 K n/a positive 69 Y N positive positive

16 H negative negative 1 N N negative positive

17 H negative negative 4 N N negative negative

18 H n/a positive 87 N Y positive negative

19 H n/a positive 101 N Y positive positive

20 H negative negative 1 N N negative negative

21 K n/a positive 5 N N positive negative

22 H n/a positive 25 N N positive positive

23 H negative negative 10 N N negative negative

24 H negative negative 5 N N negative negative

25 K positive positive 286 N Y positive positive

26 H n/a positive 44 Y N positive positive

27 H negative negative 17 N N negative positive

28 H negative negative 1 N N negative positive

29 K negative negative 19 N N negative negative

30 H positive positive 61 N Y positive positive

31 K negative negative n/a N N negative negative

32 H negative negative <1 N N negative negative

33 H negative negative 6 N N negative negative

. CC-BY-NC 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. was not certified by peer review)

(whichThe copyright holder for this preprint this version posted August 17, 2019. .https://doi.org/10.1101/19004473doi: medRxiv preprint

Page 8: Evaluation of a rapid lateral flow calprotectin test for ... · 63 intraoperative findings with selected inflammatory markers and culture results. MSIS criteria are 64 useful but

34 K positive positive 91 N N positive positive

35 H negative negative 5 N N negative positive

36 H negative negative 8 N N negative positive

37 H positive positive 114 N Y positive positive

38 K negative negative <1 N N negative negative

39 H negative negative 3 N N negative negative

40 H negative negative 1 N N negative negative

41 K negative negative 29 N N negative negative

42 H n/a positive 18 N N positive positive

43 H negative negative 1 N N negative negative

44 H negative negative 4 N N negative negative

45 H negative negative 2 N N negative negative

46 K n/a positive 219 N N positive positive

47 H negative negative <1 N N negative negative

48 H negative positive <1 N N positive negative

49 H negative negative 3 N N negative negative

50 K negative positive 4 N N positive negative

51 H positive negative 36 N Y positive positive

52 H negative negative 2 N N negative negative

53 H negative negative <1 N N negative positive

54 K negative negative 2 N N negative negative

55 K n/a positive 31 N N positive positive

56 H negative positive 30 N N positive negative

57 K negative positive 1 N N positive negative

58 H negative negative <1 N N negative negative

59 H n/a positive 49 N Y positive positive

60 K n/a positive 67 N Y positive positive

61 H negative negative 3 N N negative negative

62 H n/a positive 284 Y Y positive positive

63 H negative negative 15 N N negative negative

64 H negative negative 4 N N negative negative

65 H negative negative 3 N N negative positive

66 K negative negative 6 N N negative negative

67 H positive negative 15 N N negative positive

68 H positive negative 3 N N negative negative

69 H n/a positive 51 Y N positive positive

Table 2. MSIS and calprotectin test results. Histology: positive/negative for signs of inflammation 136 by frozen section microscopy. Culture: positive/negative for two or more cultures of the same 137 organism from periprosthetic tissue and fluid samples. CRP: positive when serum CRP level ≥10 138 mg/L. Sinus tract: Y = presence of sinus tract. Purulence: Y = visible wound purulence. MSIS: 139 positive/negative according to scoring in table 1. Calprotectin: positive when ≥ 14 mg/L synovial 140 calprotectin measured by Lyfstone calprotectin test. 141

142

A total of 24 cases were classed as infected and 45 cases aseptic according to MSIS criteria. Of the 143

infected cases, 21 were positive by routine microbiology alone (>2 cultures positive cultures for the 144

. CC-BY-NC 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. was not certified by peer review)

(whichThe copyright holder for this preprint this version posted August 17, 2019. .https://doi.org/10.1101/19004473doi: medRxiv preprint

Page 9: Evaluation of a rapid lateral flow calprotectin test for ... · 63 intraoperative findings with selected inflammatory markers and culture results. MSIS criteria are 64 useful but

same organism). Two of the remaining cases were positive by histology, elevated serum CRP and one 145

positive culture (Collinsella aeofasciens and ‘Diptheroids’) and the remaining infected case was 146

positive by a combination of histology, CRP and purulence (Table 2). 147

In this infected group there were 18 true positive and 6 false negative results by calprotectin. The 148

reported organisms in these false negative cases were Cutibacterium acnes (n=3), Bacteroides fragilis 149

(n=1), Staphylococcus epidermidis (n=1), and a polymicrobial infection of Pseudomonas aeruginosa 150

with Corynebacterium tuberculostearicum (n=1). 151

The remaining 45 cases were aseptic by MSIS criteria and of these 34 were true negative and 11 were 152

false positive by calprotectin. Of the 11 false positive cases, one case was positive for inflammation 153

by histology and elevated CRP and one was CRP positive without significant histology. The 154

remaining nine false positive samples were negative for any MSIS criteria tested. 155

In the hip revision surgery group (n=52) the test had a sensitivity of 80% and specificity of 70.3%. 156

The PPV and NPV were 52.2% and 89.7% respectively with an overall test accuracy of 73.1%. All 11 157

false positive results in the study were hip revisions. In the knee revision surgery group (n=17) the test 158

was 66.7% sensitive and 100% specific. With no false positive results, the PPV was 100% but the 159

NPV was 72.7% (Supplementary table 1). Overall test accuracy was higher than in the hip group at 160

82.4%. 161

162

Lyfstone test performance compared to MSIS+ gold standard 163

Overall test accuracy compared to the MSIS+ standard was 82.6%. Sensitivity and specificity were 164

94.7% and 78% respectively, with PPV of 62.1% and NPV of 97.5% (Table 3). 165

166

167

168

. CC-BY-NC 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. was not certified by peer review)

(whichThe copyright holder for this preprint this version posted August 17, 2019. .https://doi.org/10.1101/19004473doi: medRxiv preprint

Page 10: Evaluation of a rapid lateral flow calprotectin test for ... · 63 intraoperative findings with selected inflammatory markers and culture results. MSIS criteria are 64 useful but

169

Gold standard

MSIS MSIS+

Lyfstone test All samples Hips Knees All samples Hips Knees

Sensitivity

(95% CI)

75.00%

(53.29% - 90.23%)

80%

(51.91% - 95.67%)

66.67%

(29.93% - 92.51%)

94.74%

(73.97% - 99.87%)

92.31%

(63.97% - 99.81%)

100%

(54.07% - 100%)

Specificity

(95% CI)

75.56%

(60.46% - 87.12%)

70.27%

(53.02% - 84.13%)

100%

(63.06% - 100%)

78.00%

(64.04% - 88.47%)

71.79%

(55.13% - 85%)

100%

(71.51% - 100%)

PPV

(95% CI)

62.07%

(48.23% - 74.19%)

52.17%

(38.48% - 65.55%)

100%

(100% - 100%)

62.07%

(49% - 73.6%)

52.17%

(39.23% - 64.83%)

100%

(100% - 100%)

NPV

(95% CI)

85.00%

(73.54% - 92.04%)

89.66%

(75.51% - 96.06%)

72.73%

(51.42% - 87.04%)

97.50%

(85.2% - 99.62%)

96.55%

(80.83% - 99.47%)

100%

(100% - 100%)

Accuracy

(95% CI)

75.36%

(63.51% - 84.95%)

73.08%

(58.98% - 84.43%)

82.35%

(56.57% - 96.2%)

82.61%

(71.59% - 90.68%)

76.92%

(63.16% - 87.47%)

100%

(80.49% - 100%)

Table 3. Performance of calprotectin test on tested synovial fluid samples. All samples, n=69; Hips, n=52; Knees, n=17170

. CC-BY-NC 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. was not certified by peer review)

(whichThe copyright holder for this preprint this version posted August 17, 2019. .https://doi.org/10.1101/19004473doi: medRxiv preprint

Page 11: Evaluation of a rapid lateral flow calprotectin test for ... · 63 intraoperative findings with selected inflammatory markers and culture results. MSIS criteria are 64 useful but

171

In the hip group, medical record and radiological review revealed 7/11 false positive results were 172

associated with metallosis from patients who were being revised for metal on metal hip implants, an 173

expected trigger for gross inflammation [16] thereby resulting in high calprotectin and false positive 174

results. The remaining four false positive cases had aseptic loosening listed as the initial indication for 175

operation. Review of pre-operative radiographs revealed gross osteolysis in 3 out of 4 cases, with 176

gross destruction of either proximal femur or acetabulum. This again is likely to cause false positive 177

calprotectin results. One-year follow-up of one of the three patients (patient 35) revealed that they 178

were still experiencing pain in their joint and their CRP had raised from 1 mg/L to 19 mg/L 179

(Supplementary table 1), indicating possible infection, but as yet still under investigation by the 180

treating teams. Removal of metallosis and severe osteolysis cases from the dataset improved the 181

specificity, PPV and overall accuracy of the test both against MSIS and MSIS+ (Table 4). 182

183

Gold Standard

Lyfstone test

MSIS

(Excluding metallosis and severe

osteolysis cases)

MSIS+

(Excluding metallosis and severe

osteolysis cases)

Sensitivity

(95% CI)

71.43%

(47.82% - 88.72%)

93.75%

(69.77% - 99.84%)

Specificity

(95% CI)

96.88%

(83.78% - 99.92%)

97.30%

(85.84% - 99.93%)

PPV

(95% CI)

93.75%

(68.14% - 99.06%)

93.75%

(68.36% - 99.05%)

NPV

(95% CI)

83.78%

(72.37% - 91.06%)

93.70%

(84.36% - 99.59%)

Accuracy

(95% CI)

86.79%

(74.66% - 94.52%)

96.23%

(87.02% - 99.54%)

Table 4. Performance of calprotectin test on tested synovial fluid samples, excluding metallosis 184 and severe osteolysis cases. MSIS (excluding metallosis and severe osteolysis cases): calprotectin 185 result against MSIS criteria (n=53) for infection excluding all samples associated with metallosis and 186 severe osteolyisis. MSIS+ (excluding metallosis and severe osteolysis cases): calprotectin result 187

. CC-BY-NC 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. was not certified by peer review)

(whichThe copyright holder for this preprint this version posted August 17, 2019. .https://doi.org/10.1101/19004473doi: medRxiv preprint

Page 12: Evaluation of a rapid lateral flow calprotectin test for ... · 63 intraoperative findings with selected inflammatory markers and culture results. MSIS criteria are 64 useful but

against MSIS criteria (n=53) for infection with discrepant samples investigated by clinical follow-up 188 excluding all samples associated with metallosis and severe osteolyisis. 189

190

The three false negative results within the hip subset were all MSIS positive by culture alone (two or 191

more cultures of the same organism). Patient 18 was culture positive for B. fragilis, isolated from all 192

cultured tissue samples, along with a high CRP and noted joint purulence. Calprotectin was recorded 193

at 13.36 mg/L which falls just below the threshold value for a positive result (≥14 mg/L) 194

(Supplementary table 1). Sample 48 was culture positive for C. acnes but negative by both histology 195

and CRP. The calprotectin test was negative with 0.0 mg/L. The initial indication for operation for 196

this case was aseptic loosening and a single stage revision was performed - the C. acnes report was 197

dismissed as contamination and the patient was not treated for infection. One-year follow-up of the 198

patient showed no signs of infection indicating that the calprotectin result correlated with the clinical 199

findings. Sample 56 was a similar case of an aseptic loosening revision yielding a significant culture 200

results (P. aeruginosa and Corynebacterium) but no detectable synovial fluid calprotectin. The patient 201

had a single stage revision and upon a two-month follow-up, showed no sign of infection, indicating a 202

true negative calprotectin result. The clinical review (MSIS+) increases the sensitivity of the Lyfstone 203

test from 75% to 94.74% (Table 3). 204

In the knee group, three false negative results were recorded; two cases of C. acnes and one case 205

reporting S. epidermidis. Both C. acnes cases (Samples 50 and 57) were patients displaying no pre-206

operative indication of infection and were treated as aseptic loosening. The clinical review found no 207

complications after patient follow up suggesting both cases were aseptic and the calprotectin results 208

were correct. Sample 21 (S. epidermidis) was taken during the second stage of revision surgery 209

performed one year after the first stage operation. Clinical review revealed the patient was originally 210

treated for gross infection. At the time of the second stage revision, the patient showed no overt signs 211

of infection, so treatment progressed to implantation of the prosthesis. Post-operative microbiology 212

grew a culture of S. epidermidis from 3/6 samples taken at the time of surgery. At the one-year 213

follow-up the patient presented no sign of infection, confirming the calprotectin result to be correct, 214

. CC-BY-NC 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. was not certified by peer review)

(whichThe copyright holder for this preprint this version posted August 17, 2019. .https://doi.org/10.1101/19004473doi: medRxiv preprint

Page 13: Evaluation of a rapid lateral flow calprotectin test for ... · 63 intraoperative findings with selected inflammatory markers and culture results. MSIS criteria are 64 useful but

thus far. Overall, the Lyfstone test was 100% accurate for the diagnosis of PJI in knee samples (n=17 215

– Tables 2 and 3). 216

217

Discussion 218

PJI is a serious complication of total joint arthroplasty with associated patient morbidity and the 219

economic burden of treatment. Microbiological culture techniques are slow and insensitive and 220

although new diagnostic tests are being developed, none of the current methods are capable of reliably 221

diagnosing PJI. 222

Rapid biomarker-based tests have the potential to assist clinicians in making a pre-operative 223

diagnosis. Early differentiation of PJI from aseptic loosening may reduce the number of unnecessary 224

two-stage revisions performed on patients with a presumptive diagnosis of low-grade infection. 225

Zimmer Biomet have developed the Synovasure® Test, which functions very similarly to the 226

Lyfstone calprotectin test, measuring another neutrophil released antimicrobial peptide, α-defensin. 227

Studies using Synovasure® have shown varying results with sensitivities ranging from 67% to 100% 228

and specificities ranging from 82.4% to 100% against the MSIS criteria [17-25]. A systematic review 229

summarising the results of seven prospective trials found the Synovasure® lateral flow test had an 230

overall sensitivity of 85% and a specificity of 90% [26]. 231

The performance of the Lyfstone calprotectin rapid test was determined using MSIS and MSIS+ 232

standards. Comparing the results to the MSIS criteria, the PPV was 62% and NPV was 85%. Further 233

analysis revealed that the low PPV is heavily affected by the presence of metallosis cases from 234

revision of metal on metal implants within our sample set. Removing these samples yields a PPV of 235

79%. The use of biomarker based diagnostic tests is known to be unreliable in patients with metal-on-236

metal implants as it can cause gross inflammation producing false positive results [27]. This effect 237

was observed in our study with seven of 13 metal-on-metal revisions producing false positive results 238

(true positive = 3, true negative = 3). Patients with evidence of severe osteolysis also contributed a 239

. CC-BY-NC 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. was not certified by peer review)

(whichThe copyright holder for this preprint this version posted August 17, 2019. .https://doi.org/10.1101/19004473doi: medRxiv preprint

Page 14: Evaluation of a rapid lateral flow calprotectin test for ... · 63 intraoperative findings with selected inflammatory markers and culture results. MSIS criteria are 64 useful but

disproportionate number of false positive results (n=3). Removing both metallosis and osteolysis 240

cases resulted in a PPV of 93.8%. 241

Compared to MSIS, six false negative results were observed. Three of these were MSIS positive by 242

culture of C. acnes. This organism is a common lab contaminant and not typically considered a cause 243

of infection when isolated from hip and knee tissues [28-30]. In fact, 5/6 false negative Lyfstone test 244

results were shown to be true negatives when compared to the MSIS+ clinical standard, highlighting 245

the need for improved diagnostics. Only one case of infection was missed (case 18) in which the 246

calprotectin result was very close to positive (13.36mg/L, < 1 mg/L below the positive cut-off). If 247

future optimisation of the test sets a lower threshold value for infection, then this case would have 248

been classified as positive. Such a readjustment of the calprotectin cut-off values may improve test 249

performance in detecting low-grade infections. 250

The sensitivity of the Lyfstone test increased from 75% to 94.74% when using the MSIS+ compared 251

to the MSIS criteria and the NPV increased from 85% to 97.5%. There was only a modest increase in 252

specificity and no increase in PPV. However, excluding both metallosis and osteolysis patients 253

increased specificity to 97.3% and PPV to 93.75%. 254

Test accuracy was different for hip and knee revisions. False positive results were only observed in 255

hip revisions, not in knee revision as there was no metal-on-metal reactions or significant osteolysis in 256

the knee revision group. Compared to MSIS+, the Lyfstone test was 100% accurate for the diagnosis 257

of PJI in knee revisions, however the sample size is small (n=17) and more data is required to confirm 258

these findings. 259

One limitation of this study is the use of retrospective samples, however, by testing all eligible 260

samples in the biorepository we believe that we did not introduce any sample selection bias. Another 261

limitation was the samples had been frozen for storage before testing and the freeze-thaw process may 262

result in leukocyte cell lysis and an increase in calprotectin. As the test is validated for use on fresh 263

synovial fluid samples, the recommended cut-offs may not necessarily be appropriate for our samples. 264

. CC-BY-NC 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. was not certified by peer review)

(whichThe copyright holder for this preprint this version posted August 17, 2019. .https://doi.org/10.1101/19004473doi: medRxiv preprint

Page 15: Evaluation of a rapid lateral flow calprotectin test for ... · 63 intraoperative findings with selected inflammatory markers and culture results. MSIS criteria are 64 useful but

It is worth noting that blood contamination of the samples had no impact on the accuracy of the test 265

results, something the Synovasure® lateral flow test warns may lead to false negative results [31]. 266

In conclusion, the Lyfstone calprotectin assay shows good potential as a rapid diagnostic test for PJI 267

due to its high sensitivity and NPV. The test has a low PPV compared to MSIS and MSIS+ but the 268

false positive results are mostly in patients with metal on metal reactions or significant osteolysis, 269

which could serve as contraindications for use. Prospective studies on fresh samples in appropriate 270

patient populations are required to accurately define test performance. 271

272

Funding statement 273

This paper presents independent research funded by Orthopaedic Research UK (ORUK – grant 274

number 526) the Biotechnology and Biological Sciences Research Council (BBSRC) Institute 275

Strategic Programme Microbes in the Food Chain BB/R012504/1 and its constituent projects 276

BBS/E/F/000PR10348 and BBS/E/F/000PR10349 (JOG). 277

278

Author contributions 279

Research design – AT, JM, IM, JOG; Sample testing – AT, RD, RBS; Clinical data acquisition – RD, 280

CW, RBS; Identification, consent and collection of samples – CW, CH, MS, AD; Patient reviews – 281

IM; Manuscript writing – AT, RD, GK, IM, JOG; Training and support – JM; Clinical management 282

and setup of trial – IM, JW; Manuscript review – all authors 283

284

Competing interests 285

JM is Chief Scientific Officer at Lyfstone. Lyfstone supplied calprotectin lateral flow tests free of 286

charge for the study. IM is a paid consultant for Lyfstone. 287

288

. CC-BY-NC 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. was not certified by peer review)

(whichThe copyright holder for this preprint this version posted August 17, 2019. .https://doi.org/10.1101/19004473doi: medRxiv preprint

Page 16: Evaluation of a rapid lateral flow calprotectin test for ... · 63 intraoperative findings with selected inflammatory markers and culture results. MSIS criteria are 64 useful but

References 289

1. HQIP. National Joint Registry 15th Annual Report (2018). 28 May 2019]; Available from: 290 https://www.hqip.org.uk/resource/national-joint-registry-15th-annual-report-2018/. 291

2. Tande, A.J. and R. Patel, Prosthetic joint infection. Clin Microbiol Rev, 2014. 27(2): p. 302-292 45. 293

3. Pérez-Prieto, D., et al., C-reactive protein may misdiagnose prosthetic joint infections, 294 particularly chronic and low-grade infections. International Orthopaedics, 2017. 41(7): p. 295 1315-1319. 296

4. Siedlecki, C., et al., Complications and cost of single-stage vs. two-stage bilateral 297 unicompartmental knee arthroplasty: A case-control study. Orthopaedics & Traumatology: 298 Surgery & Research, 2018. 104(7): p. 949-953. 299

5. Parvizi, J., et al., New definition for periprosthetic joint infection: from the Workgroup of the 300 Musculoskeletal Infection Society. Clin Orthop Relat Res, 2011. 469(11): p. 2992-4. 301

6. Parvizi, J., et al., The 2018 Definition of Periprosthetic Hip and Knee Infection: An Evidence-302 Based and Validated Criteria. The Journal of Arthroplasty, 2018. 33(5): p. 1309-1314.e2. 303

7. Striz, I. and I. Trebichavsky, Calprotectin - a pleiotropic molecule in acute and chronic 304 inflammation. Physiol Res, 2004. 53(3): p. 245-53. 305

8. Witter, A.R., B.M. Okunnu, and R.E. Berg, The Essential Role of Neutrophils during 306 Infection with the Intracellular Bacterial Pathogen Listeria monocytogenes. J Immunol, 2016. 307 197(5): p. 1557-65. 308

9. van Rheenen, P.F., E. Van de Vijver, and V. Fidler, Faecal calprotectin for screening of 309 patients with suspected inflammatory bowel disease: diagnostic meta-analysis. BMJ, 2010. 310 341: p. c3369. 311

10. Abildtrup, M., G.H. Kingsley, and D.L. Scott, Calprotectin as a Biomarker for Rheumatoid 312 Arthritis: A Systematic Review. The Journal of Rheumatology, 2015. 42(5): p. 760-770. 313

11. Wouthuyzen-Bakker, M., et al., Synovial Calprotectin: An Inexpensive Biomarker to Exclude 314 a Chronic Prosthetic Joint Infection. The Journal of Arthroplasty, 2018. 33(4): p. 1149-1153. 315

12. Lyfstone, LYFCLP005 Lateral Flow Test Kit Instructions For Use. 316 13. England, P.H. UK Standards for Microbiology Investigations: Investigation of orthopaedic 317

implant associated infections. 2016; Available from: 318 https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/319 file/504319/B_44i2.pdf. 320

14. Service, U.K.A. UKAS Medical ISO 8405. 2019; Available from: https://www.ukas.com/wp-321 content/uploads/schedule_uploads/00007/8405%20Medical%20Single.pdf. 322

15. Service, U.K.A., UKAS Medical ISO 10294. 2018. 323 16. Drummond, J., P. Tran, and C. Fary, Metal-on-Metal Hip Arthroplasty: A Review of Adverse 324

Reactions and Patient Management. J Funct Biomater, 2015. 6(3): p. 486-99. 325 17. Berger, P., et al., Diagnosis of prosthetic joint infection with alpha-defensin using a lateral 326

flow device: a multicentre study. Bone Joint J, 2017. 99-b(9): p. 1176-1182. 327 18. Bingham, J., et al., The alpha defensin-1 biomarker assay can be used to evaluate the 328

potentially infected total joint arthroplasty. Clin Orthop Relat Res, 2014. 472(12): p. 4006-9. 329 19. Eriksson, H.K., et al., Does the Alpha-defensin Immunoassay or the Lateral Flow Test Have 330

Better Diagnostic Value for Periprosthetic Joint Infection? A Systematic Review. Clin Orthop 331 Relat Res, 2018. 476(5): p. 1065-1072. 332

20. Frangiamore, S.J., et al., alpha-Defensin Accuracy to Diagnose Periprosthetic Joint Infection-333 Best Available Test? J Arthroplasty, 2016. 31(2): p. 456-60. 334

21. Gehrke, T., et al., The Accuracy of the Alpha Defensin Lateral Flow Device for Diagnosis of 335 Periprosthetic Joint Infection: Comparison with a Gold Standard. J Bone Joint Surg Am, 336 2018. 100(1): p. 42-48. 337

22. Kasparek, M.F., et al., Intraoperative Diagnosis of Periprosthetic Joint Infection Using a 338 Novel Alpha-Defensin Lateral Flow Assay. J Arthroplasty, 2016. 31(12): p. 2871-2874. 339

. CC-BY-NC 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. was not certified by peer review)

(whichThe copyright holder for this preprint this version posted August 17, 2019. .https://doi.org/10.1101/19004473doi: medRxiv preprint

Page 17: Evaluation of a rapid lateral flow calprotectin test for ... · 63 intraoperative findings with selected inflammatory markers and culture results. MSIS criteria are 64 useful but

23. Riccio, G., et al., Qualitative Alpha-defensin Versus The Main Available Tests For The 340 Diagnosis Of Periprosthetic Joint Infection: Best Predictor Test? J Bone Jt Infect, 2018. 3(3): 341 p. 156-164. 342

24. Sigmund, I.K., et al., Is the Enzyme-linked Immunosorbent Assay More Accurate Than the 343 Lateral Flow Alpha Defensin Test for Diagnosing Periprosthetic Joint Infection? Clin Orthop 344 Relat Res, 2018. 476(8): p. 1645-1654. 345

25. Stone, W.Z., et al., Clinical Evaluation of Synovial Alpha Defensin and Synovial C-Reactive 346 Protein in the Diagnosis of Periprosthetic Joint Infection. J Bone Joint Surg Am, 2018. 347 100(14): p. 1184-1190. 348

26. Marson, B.A., et al., Alpha-defensin and the Synovasure lateral flow device for the diagnosis 349 of prosthetic joint infection: a systematic review and meta-analysis. Bone Joint J, 2018. 100-350 B(6): p. 703-711. 351

27. Bonanzinga, T., et al., How Reliable Is the Alpha-defensin Immunoassay Test for Diagnosing 352 Periprosthetic Joint Infection? A Prospective Study. Clinical Orthopaedics and Related 353 Research®, 2017. 475(2): p. 408-415. 354

28. Both, A., et al., Growth of Cutibacterium acnes is common on osteosynthesis material of the 355 shoulder in patients without signs of infection. Acta Orthop, 2018. 89(5): p. 580-584. 356

29. Levy, P.Y., et al., Propionibacterium acnes Postoperative Shoulder Arthritis: An Emerging 357 Clinical Entity. Clinical Infectious Diseases, 2008. 46(12): p. 1884-1886. 358

30. Patel, A., et al., Propionibacterium acnes colonization of the human shoulder. Journal of 359 Shoulder and Elbow Surgery, 2009. 18(6): p. 897-902. 360

31. Diagnostics, C. Synovasure® Alpha Defensin Lateral Flow Test Kit Instructions For Use. 361 01/08/2019]; Available from: 362 https://cddiagnostics.com/instructions/PDF/LF%20Test%20Kit%20IFUs/EN_M40004B_V6_363 Synovasure_Alpha_Defensin_Lateral_Flow_IFU.pdf. 364

365

. CC-BY-NC 4.0 International licenseIt is made available under a is the author/funder, who has granted medRxiv a license to display the preprint in perpetuity. was not certified by peer review)

(whichThe copyright holder for this preprint this version posted August 17, 2019. .https://doi.org/10.1101/19004473doi: medRxiv preprint