supplementary appendix - the · pdf filesupplementary appendix ... curve analysis table d c...

Supplementary appendixThis appendix formed part of the original submission and has been peer reviewed. We post it as supplied by the authors.

Supplement to: Hijazi Z, Oldgren J, Lindbäck J, et al, on behalf of the ARISTOTLE and RE-LY Investigators. The novel biomarker-based ABC (age, biomarkers, clinical history)-bleeding risk score for patients with atrial fibrillation: a derivation and validation study. Lancet 2016; published online April 4. http://dx.doi.org/10.1016/S0140-6736(16)00741-8.

The ABC-bleeding risk score in atrial fibrillation Z. Hijazi et al.

1

SUPPLEMENTARY MATERIAL

A Novel Biomarker-Based Bleeding Score for Patients with Atrial Fibrillation – The ABC

(Age, Biomarkers, Clinical history) Risk Score

Z. Hijazi, et al.

TABLE OF CONTENTS

Summary of the derivation and validation cohorts

Statistical analyses (detailed)

Table A C-indices for intracranial haemorrhages for the ABC-bleeding score in comparison with

the HAS-BLED and ORBIT scores in the full cohorts and in the warfarin treated

subgroup

Table B Event rates and hazard ratios between ABC-bleeding risk classes for the derivation and

the validation cohorts

Table C Net benefit of the ABC-bleeding score as compared with the HAS-BLED and ORBIT

scores using decision curve analysis

Table D C indices for the ABC-bleeding score using haematocrit instead of haemoglobin

Figure A Calibration plots of the ABC-bleeding, ORBIT, and HAS-BLED models

Figure B 1: Kaplan-Meier estimated cumulative event rate by the three HAS-BLED risk classes

(0-1 points, 2 points, ≥3 points) for the ABC-bleeding score (panel: low, medium, and

high).

2: Kaplan-Meier estimated cumulative event rate by the three ORBIT risk classes (0-2

points, 3 points, ≥4 points) for the ABC-bleeding score (panel: low, medium, and high).

Figure C ABC-bleeding score nomogram using cardiac troponin-I high sensitivity (instead of

troponin-T high sensitivity)


2

Figure D The importance of each variable in the full multivariable model without GDF-15 as

measured by partial Wald χ2 minus the predictor degrees of freedom

Figure E The ABC-bleeding score nomogram using the biomarker cystatin-C (instead of GDF-

15)

Figure F The ABC-bleeding score nomogram using CKD-EPI (instead of GDF-15)

Figure G The ABC-bleeding score nomogram using haematocrit (instead of haemoglobin)

Figure H, I Application of the ABC-bleeding nomogram exemplified

Figure J Forest plot of the full model including all candidate variables

Figure K Forest plot of the final model

Sensitivity analysis


3

Summary of the derivation and validation cohorts

Derivation cohort

ARISTOTLE1 was a double blind, randomised clinical trial that enrolled 18,201 patients with AF at

increased risk for stroke at 1034 clinical sites in 39 countries between December 2006 and April 2010.

Patients included had paroxysmal, persistent or permanent AF, or atrial flutter, and one or more of the

following risk factors; age ≥75 years, prior stroke, transient ischemic attack (TIA), or systemic

embolus, heart failure, diabetes mellitus, or hypertension requiring pharmacologic treatment. Among

the exclusion criteria were; clinically significant mitral stenosis, mechanical heart valve, recent stroke,

previous intracranial haemorrhage, creatinine clearance less than 25 mL/min, or active alcohol or drug

abuse.1 Participants were randomised to warfarin (n=9,081) or apixaban (n=9,120). The median length

of follow-up was 1.7 years for the 14,537 participants with biomarker samples available at

randomisation after exclusion of 45 (0.3%) patients with missing data.

External validation cohort

RE-LY2 was a prospective, multicentre, randomized trial comparing two blinded doses of dabigatran

with open label warfarin that enrolled 18,113 patients with AF at 951 clinical sites in 44 countries

between December 2005 and Mars 2009. Inclusion criteria were documented atrial fibrillation and at

least one of the following risk factors for stroke: previous stroke or TIA; congestive heart failure or

reduced left ventricular ejection fraction (<40%); at least 75 years of age; or at least 65 years of age

with diabetes mellitus, hypertension, or coronary artery disease. Exclusion criteria included severe

heart valve disorder, recent stroke, creatinine clearance less than 30 mL/min, or active liver disease.

The median length of follow-up was 1.9 years for the 8,468 participants with biomarker samples

available at randomisation.


4

Statistical analyses (detailed)

Development of the prediction model in the ARISTOTLE cohort

All biomarkers were log-transformed and values below the reporting limit were set to half the limit. In

the first step a model including all candidate predictors (listed in Figure 1) was fitted. Possible non-

linearities were evaluated by transforming the continuous variables using restricted cubic splines, each

with four knots placed at the respective 5th, 35th, 65th, and 95th sample percentiles. To allow for

different prediction models for subjects randomised to warfarin or apixaban, bivariate interactions

between each variable and randomised treatment were included. The global test of any interaction was

not statistically significant so the full prediction model included only main effects. The test of any

non-linearity was statistically significant resulting in all non-linear terms being retained in the full

model. To obtain a more parsimonious and clinically useful model we approximated the full model by

using a fast backward algorithm on an ordinary least squares model in which the estimated linear

predictor from the full Cox model was the outcome and all candidate variables were entered in exactly

the same manner as in the full Cox model.3 Thus, in the first step R²=1.0 by design and by removing

variables in a stepwise manner the full model could be approximated to an arbitrary level. None of the

continuous variables included in the final model showed any sign of a non-linear relation with the

outcome so all were included as linear terms. Further, additional models were created in the same

manner however using alternative biomarkers i.e. replacing cTnT-hs with cTnI-hs, replacing

haemoglobin with haematocrit and replacing GDF-15 with cystatin-C or creatinine clearance estimated

by the CKD-EPI equation.

The final models were presented as nomograms and compared with the HAS-BLED and ORBIT

scores using Harrell's c-index. Differences between c-indices were tested using 10 000 bootstrap

samples. Risk categories were created according to <1%, 1-2%, and >2% risk for bleeding within one

year. The risk categories were selected according to clinically relevant cut-offs in patients with AF

regarding antithrombotic treatment.4

Internal and external model validation

The model was internally validated using 300 bootstrap samples. Within each bootstrap sample we

refitted the model and compared the apparent performance in the bootstrap sample with the test

performance (applying the refitted model to the original data). The optimism was quantified as the

mean difference of these performance estimates. To reduce the optimism of new predictions we

applied uniform shrinkage to the regression parameters. First, the linear predictor was calculated for

all subjects in the original sample using the estimated regression coefficients from the models fitted

within each bootstrap sample. Then, using the observed outcomes in the original sample, the slope for

the linear predictor was estimated using a Cox-regression model. The average of all slopes determined


5

the amount of shrinkage. External validation was conducted in 8,468 patients from the RE-LY trial.

Prior bleeding information was not routinely collected and was defined as anaemia at baseline,

sensitivity analyses suggested consistent results when all subjects were assumed to have no prior

bleeding.

Discrimination and calibration

Discrimination was assessed by Harrell’s c-index and by comparing Kaplan-Meier curves and hazard

ratios between the predefined risk categories. Clinical usefulness and net benefit of the predictive

models were estimated with decision curve analysis.4 Calibration was assessed by comparing one-year

event rates with predictions from the final model by fitting a Cox-regression model with the estimated

one-year event probability included as a restricted cubic spline. The final model was also evaluated in

different subgroups of the data; without a history of bleeding, in patients also on concomitant

antiplatelet or NSAID therapy, and in the group randomised to warfarin therapy. The HAS-BLED

categories were adapted from current guidelines recommendations4, and the ORBIT categories were

based on the original ORBIT publication8.

Quality of methodology

The analyses followed the framework for derivation and validation of prediction models proposed by

Harrell, and Steyerberg and Vergouwe.3, 6

The external validation followed the principles and methods

described by Royston and Altman and the reporting followed the recently published TRIPOD

statement (protocol checklist submitted).7, 8

All analyses were performed using R version 3.2 using the

packages rms and Hmisc.3


6

Table A

C-indices (95% CI) for intracranial haemorrhages for the ABC-bleeding score in comparison with the

HAS-BLED and ORBIT scores in the full cohorts and in the warfarin treated subgroup.

Risk model Full cohort Warfarin

treatment

Cohort (events/n in group) 207/23005 135/1066

ABC-bleeding 0.66 (0.62-0.69) 0.66 (0.62-0.71)

HAS-BLED 0.58 (0.54-0.61) 0.62 (0.58-0.66)

ORBIT 0.60 (0.56-0.64) 0.63 (0.58-0.68)

ABC-bleeding: Age, Biomarkers (troponin T-hs, haemoglobin, and GDF-15), Clinical history (prior bleeding)

ORBIT: Older age [75+ years], Reduced haemoglobin/haematocrit/history of anaemia, Bleeding history,

Insufficient kidney function, and Treatment with antiplatelet

HAS-BLED: Hypertension, Abnormal renal/liver function, Stroke, Bleeding history or predisposition, Labile

INR, Elderly (>65), Drugs/alcohol concomitantly)


7

Table B

Event rates and hazard ratios between ABC-bleeding risk classes for the derivation and the validation

cohorts. Numbers within parentheses indicate 95% confidence limits.

Risk class N Events Incidence rate* Hazard ratio

Derivation cohort

Low (< 1 %) 1877 13 0.36 [0.24, 0.71] 1.00 (ref)

Medium (1 – 2 %) 3958 113 1.56 [1.30, 1.89] 4.28 [2.41, 7.61]

High (> 2 %) 8702 536 3.75 [3.38, 4.01] 10.25 [5.91, 17.8]

Validation cohort

Low (< 1 %) 1179 15 0.62 [0.36, 1.10] 1.00 (ref)

Medium (1 – 2 %) 3554 117 1.67 [1.26, 1.87] 2.66 [1.55, 4.55]

High (> 2 %) 3735 331 4.87 [4.34, 5.40] 7.63 [4.55 12.81]

*per 100 person-years


8

Table C

Net benefit of using the ABC, ORBIT and HAS-BLED scores for identifying major bleedings

conditional on different decision thresholds. The three scores are compared with the assumption that

no one is at risk for major bleeding (all negative) and the assumption that all are at risk for major

bleeding (all positive). The numbers indicate the number of additional true positives per 1000 person-

years the models can identify without additional false positives.

Decision

Threshold (%)

Net benefit vs all negative Net benefit vs all positive

All positive ABC ORBIT HAS-BLED ABC ORBIT HAS-BLED

1 21.1 21.9 21.3 21.1 0.7 0.2 0.0

2 11.1 14.5 13.5 5.7 3.3 2.3 -5.5

3 1.0 9.2 7.8 4.2 8.2 6.9 3.3

4 -9.5 6.0 5.3 0.9 15.4 14.7 10.3

5 -20.1 4.0 3.2 0.7 24.1 23.3 20.8

6 -30.9 2.8 1.9 0.4 33.8 32.8 31.4

The net benefit was estimated using a decision curve analysis5 on the 23,005 subjects with an event

rate of 27.2 major bleedings per 1000 person-years in the combined data from the ARISTOTLE and

RE-LY studies.


9

Table D

C indices for the ABC-bleeding score using haematocrit instead of haemoglobin

Risk model Full cohort No previous

bleeding

Warfarin

treatment

NOAC

treatment*

Antiplatelet or

NSAID†

Derivation cohort (events/n in group) 662/14537 515/12164 386/7252 276/7285 321/5609

ABC-bleeding (haematocrit) 0.68

(0.66 - 0.70)

0.68

(0.65 - 0.70)

0.68

(0.65 - 0.70)

0.68

(0.65 - 0.71)

0.69

(0.66 - 0.72)

Validation cohort (events/n in group) 447/8152 323/7084 155/2709 292/5443 243/3540

ABC-bleeding (haematocrit) 0.70

(0.68 – 0.73)

0.68

(0.65 - 0.71)

0.65

(0.61 - 0.70)

0.73

(0.70 - 0.76)

0.67

(0.64 - 0.71)

*Apixaban in the derivation cohort and dabigatran in the validation cohort. †Concomitant use of

antiplatelet therapy (aspirin or clopidogrel) or non-steroidal anti-inflammatory drugs.


10

Figure A

Calibration plots of observed vs. predicted event rate for the HAS-BLED (left), ORBIT (middle), and

ABC-bleeding (right) risk models in the validation and derivation cohorts.

Major bleeding events rates per 100 patient-years (95% CI) observed in the ARISTOTLE and RE-LY trials vs.

the previously published event rates from the original derivation cohorts for the ORBIT and HAS-BLED

scores.9, 10


11

Figure B

1: Kaplan-Meier estimated cumulative event rate by the three HAS-BLED risk classes (0-1 points, 2

points, ≥3 points) for the ABC-bleeding score (panel: low, medium, and high).

2: Kaplan-Meier estimated cumulative event rate by the three ORBIT risk classes (0-2 points, 3 points,

≥4 points) for the ABC-bleeding score (panel: low, medium, and high).


12

Figure C

ABC-bleeding score nomogram using cardiac troponin-I high sensitivity (instead of troponin-T high

sensitivity)

For each predictor, read the points assigned on the 0-10 scale at the top and then add these points. Find the

number on the “Total Points” scale and then read the corresponding predictions of 1- and 3-year risk of bleeding.

Continuous variables are represented from the 1st to the 99

th percentiles.

The prediction model is preferably used as a web-based calculator or app.


13

Figure D

The importance of each variable in the full multivariable model without GDF-15 as measured by

partial Wald χ2 minus the predictor degrees of freedom.

Higher values on the x-axis indicate greater importance for the prediction of major bleeding events


14

Figure E

The ABC-bleeding score nomogram using the biomarker cystatin C (instead of GDF-15).




th percentiles.



15

Figure F

The ABC-bleeding score nomogram using CKD-EPI (instead of GDF-15).




th percentiles



16

Figure G

The ABC-bleeding score nomogram using haematocrit (instead of haemoglobin).




th percentiles



17

Figure H

Example 1: A 65-year old man with atrial fibrillation, hypertension, prior stroke, prior bleeding,

troponin T levels of 5 ng/L, GDF-15 levels of 600 ng/L, and haemoglobin 15 g/dL. By using the ABC-

bleeding score nomogram receives 3.75p for age, 1.5p for troponin levels, 1p for GDF-15 levels, 2p

for haemoglobin levels, and 2p for prior bleeding event. A total of 10.25p would equal a predicted 1-

year risk of major bleeding event below 1.0% and a 3-year risk below 3.0%.


18

Figure I

Example 2: A 61-year old patient with atrial fibrillation, heart failure, no prior bleeding, troponin T

levels of 26 ng/L, GDF-15 levels of 4000 ng/L, and haemoglobin 12 g/dL. By using the ABC-bleeding

score receives 3p for age, 0p for no prior bleeding event, 7p for troponin, 6.5p for GDF-15 levels, and

4p for haemoglobin. A total of 20p would approximately equal a predicted 1-year and 3-year risk of

major bleeding event 3.0% and 8.0%, respectively.


19

Figure J

Forest plot of the full model including all candidate variables. The filled triangles indicate the relative

hazard of major bleeding for each variable, adjusted for all other variables in the model. Horizontal

bars indicate 95% confidence intervals. For the continuous variables hazard ratios are estimated for the

third vs the first quartile. Note that all continuous variables were included as restricted cubic splines

with four knots.


20

Figure K

Forest plot of the final model. The filled triangles indicate the relative hazard of major bleeding for

each variable, adjusted for all other variables in the model. Horizontal bars indicate 95% confidence

intervals. For the continuous variables hazard ratios are estimated for the third vs the first quartile.


21

Sensitivity analyses

A model containing the clinical variables in the ABC-bleeding score (age and prior bleeding) and the

clinical variables reflecting the biomarkers in the ABC score (renal impairment; CrCl <60 mL/min

(instead of GDF-15 or cystatin C), history of anaemia (instead of haemoglobin/haematocrit),

cardiovascular dysfunction; heart failure (instead of troponin I or T) was constructed. This model

yielded a c-index of 0.64.


22

References

1. Lopes RD, Alexander JH, Al-Khatib SM, et al. Apixaban for reduction in stroke and

other ThromboemboLic events in atrial fibrillation (ARISTOTLE) trial: design and

rationale. Am Heart J 2010; 159(3): 331-9.

2. Ezekowitz MD, Connolly S, Parekh A, et al. Rationale and design of RE-LY:

randomized evaluation of long-term anticoagulant therapy, warfarin, compared with

dabigatran. Am Heart J 2009; 157(5): 805-10, 10 e1-2.

3. Harrell FE. Regression Modeling Strategies: With Applications to Linear Models,

Logistic and Ordinal Regression, and Survival Analysis. Springer, New York ISBN 978-

3-319-19424-0 2015.

4. Camm AJ, Kirchhof P, Lip GY, et al. Guidelines for the management of atrial

fibrillation: the Task Force for the Management of Atrial Fibrillation of the European

Society of Cardiology (ESC). Eur Heart J 2010; 31(19): 2369-429.

5. Vickers AJ, Cronin AM, Elkin EB, Gonen M. Extensions to decision curve analysis, a

novel method for evaluating diagnostic tests, prediction models and molecular markers.

BMC medical informatics and decision making 2008; 8: 53.

6. Steyerberg EW, Vergouwe Y. Towards better clinical prediction models: seven steps for

development and an ABCD for validation. Eur Heart J 2014; 35(29): 1925-31.

7. Moons KG, Altman DG, Reitsma JB, et al. Transparent Reporting of a multivariable

prediction model for Individual Prognosis Or Diagnosis (TRIPOD): Explanation and

Elaboration. Ann Intern Med 2015; 162(1): W1-W73.

8. Royston P, Altman DG. External validation of a Cox prognostic model: principles and

methods. BMC medical research methodology 2013; 13: 33.

9. O'Brien EC, Simon DN, Thomas LE, et al. The ORBIT bleeding score: a simple bedside

score to assess bleeding risk in atrial fibrillation. Eur Heart J 2015.

10. Pisters R, Lane DA, Nieuwlaat R, de Vos CB, Crijns HJ, Lip GY. A novel user-friendly

score (HAS-BLED) to assess one-year risk of major bleeding in atrial fibrillation

patients: The Euro Heart Survey. Chest 2010; (138(5)): 1093-100.

supplementary appendix - the · pdf filesupplementary appendix ... curve analysis table d c...

Documents