visit us on

https://www.chemie.uni-hamburg.de

Handling inter-occasion variability in model implementation for Bayesian forecasting: A comparison of methods and metrics.

Sebastian G. Wicha (1) and Stefanie Hennig (2)(1) Dept. of Clinical Pharmacy, Institute of Pharmacy, University of Hamburg, Germany

(2) School of Pharmacy, University of Queensland, Brisbane, Australia

Literature[1] Llanos-Paez ET AL. Antimicrob Agents Chemother. 2017;61(8).[2] Abrantes et al. PAGE 26 Abstr 7290 (2017).

IntroductionWhen implementing a PK model into aBayesian forecasting (BF) program fortherapeutic drug monitoring (TDM), theimplantation of inter-occasion variability (IOV)can greatly impact on the predictions. Anumber of approaches exist to handle IOV,which include ignoring IOV, weightingfunctions, or variations of accounting for IOVduring Bayesian estimation. In this study, weaimed to compare five methods for handlingIOV using different metrics in simulations andin a real dataset example.

MethodsSimulations were performed using a 1-compartment population PK model (CL: 5 L/h,V: 20 L, interindividual variabilities (variance)IIVCL: 0.1, IIVV: 0.1, varying IOVCL: 0.0-0.1,proportional unexplained residual variability(RUVprop) 10 %CV) with a rich (8 samples over 8-hourly dosing) and a sparse sampling design (2samples at 1 h and 7 h post dose) in 1000subjects. A real dataset (423 patients, 2422 PKsamples) and the resultant 2 compartmental PKmodel [1] was also utilised.Forecasting of occasion 6 PK observations forevery individual using data from occasions 1-5(simulation study) or of occasion 3 PKobservations from occasions 1-2 data (real data)was assessed. Simulations, estimations andforecasting was performed in NONMEM® 7.4.1.The model implementation methods testedwere:(i) ‘True’ model with IIV and IOV, quantifying

ηIIV and ηIOV’s, but using only ηIIV forforecasting [2]

(ii) IIV + IOV: adding ω²IOV to ω²IIV together(iii) IIV-only 1: re-estimation of a model without

IOV, using the new parameters forforecasting

(iv) IIV-only 2: setting ω²IOV to zero(v) IIV-only 3: weighting down samples from

past occasions by doubling RUV of theseThe metrics to evaluate forecasting were:A rBias/rRMSE calculated based on the

individual predicted (IPRED) vs. observedconcentration (DV) at the forecasted dosingoccasion, and

B rBias/rRMSE calculated based on theestimated individual PK parameter (EBEwithout ηIOV) versus the true parameter(simulation study) or the individual PKparameter determined from the finalpublished model (real data).

ResultsThe simulation study showed that increasingIOV increased rBias/rRMSE in all metrics (Fig.1 and 2). Metrics A displayed a positive bias inall scenarios with method (v) being leastbiased, followed by (i), (iii), and (ii)≈(iv).For metrics B, individual CL and V determinedby method (i) showed to be least biased,followed by (iii), (iv), (ii), and (v). Similarresults were obtained with the sparsesimulation data (Fig. 2)

ConclusionMetrics A, although popular and frequentlyused, was intrinsically biased in presence ofIOV and hence should be interpreted withcaution. As a consequence, metrics A wronglysuggested the weighting approach (v) tooutperform the true model (i) in thesimulation study.Comparisons on the forecasting performanceof models on the level of estimated vs. trueindividual PK parameters, i.e. metrics B mightbe more meaningful, but susceptible toshrinkage (reason for not showing V in the realclinical dataset). Similar trends in forecastingaccuracy were observed in the simulationstudy and the real dataset, but less marked inthe latter.Overall, method (i) displayed the bestforecasting performance. Method (iii), whereIOV was not estimated may be preferable overthe weighting method (v) in presence of IOV.

Figure 1: Bias (upper panel) and RMSE (lower panel, log-scale) of IOV handling methods (i)-(v) evaluated by IPRED-DV based metrics A and EBE-based metrics B for thesimulation study with the rich design.

Contact information

Sebastian G. Wicha, PhD sebastian.wicha@uni-hamburg.deStefanie Hennig, PhD s.hennig@uq.edu.au

The intrinsic bias observed in metrics A mightoriginate from the ‘overlay’ of theproportional residual variability with the log-normal distribution of the IOV component(Fig. 4).

i ii iii iv v

0 0.01 0.02 0.1 0 0.01 0.02 0.1 0 0.01 0.02 0.1 0 0.01 0.02 0.1 0 0.01 0.02 0.1

0

20

40

60

80

ω²IOV

Bia

s [%

]

Metrics

A_DV

B_CL

B_V

i ii iii iv v

0 0.01 0.02 0.1 0 0.01 0.02 0.1 0 0.01 0.02 0.1 0 0.01 0.02 0.1 0 0.01 0.02 0.1

1

10

100

ω²IOV

RM

SE

[%

] Metrics

A_DV

B_CL

B_V

i ii iii iv v

0 0.01 0.02 0.1 0 0.01 0.02 0.1 0 0.01 0.02 0.1 0 0.01 0.02 0.1 0 0.01 0.02 0.1

10

1000

ω²IOV

RM

SE

[%

] Metrics

A_DV

B_CL

B_V

0

10

20

30

i ii iii iv v

IOV method

Bia

s [%

] Metrics

A_DV

B_CL

1

10

100

i ii iii iv v

IOV method

RM

SE

[%

] Metrics

A_DV

B_CL

Figure 2: Bias (upper panel) and RMSE (lower panel, log-scale) of IOV handling methods (i)-(v) evaluated by IPRED-DV based metrics A and EBE-based metrics B for thesimulation study with the sparse design.

Figure 3: Bias (left) and RMSE (right) of IOV handlingmethods (i)-(v) evaluated by IPRED-DV based metrics A andEBE-based metrics B for the real dataset of gentamicin.

0 20 40 60 80 100

-30

-20

-10

01

02

03

0

Residual distribution at occasion 6

IPRED

Re

sid

ua

l

0 20 40 60 80

-15

-10

-50

51

01

5

Residual distribution at occasion 1

IPRED

Re

sid

ua

l

Using real data (Fig. 3), metrics A alsodisplayed a positive bias in all scenarios.Method (v) was least biased, followed by (i),(iii), (iv), and (ii).For metrics B, method (i) showed to be leastbiased, followed by (ii), (iv), (v) and (iii).

Figure 4: Residual distribution at occasion 1 (DV used toobtain individual PK parameters) and occasion 6 (DV fromoccasions 1-5 used to determine individual PK parameter,occasion 6 forecasted) obtained in the simulation studywith ω²IOV of 0.1.i ii iii iv v

0 0.01 0.02 0.1 0 0.01 0.02 0.1 0 0.01 0.02 0.1 0 0.01 0.02 0.1 0 0.01 0.02 0.1

0

40

80

120

ω²IOV

Bia

s [%

]

Metrics

A_DV

B_CL

B_V