take it to the limit: quantitation, likelihood, modelling and other matters

(c) Senn 2015 1Take it to the limit 1

Take it to the limit: quantitation, likelihood, modelling, inference

and other mattersStephen Senn

Acknowledgements

(c) Senn 2015 2

Acknowledgements

Thanks to Gavin Jarvis and the BPS for inviting me

This work is partly supported by the European Union’s 7th Framework Programme for research, technological development and demonstration under grant agreement no. 602552. “IDEAL”

It is also based on collaborative work with Nick Holford and Hans Hockey which was published as ‘The Ghosts of Departed Quantities’ in Statistics in Medicine (2012)

What I hope to do here

• Take a classic problem relevant to PK• Apply a standard statistical technique

– The method of maximum likelihood• Illuminate one with the other

– And vice versa• Show that simulation is not always

necessary to compare methods

Take it to the limit (c) Senn 2015 3


Background• A common requirement in developing an

assay is to establish a ‘lower limit of quantitation’ (LLOQ) or ‘quantification’

• Values below this limit are flagged as being unreliable (below limit of quantitation = BLQ) and it is often suggested that they should not be used

• This then raises various issues regarding estimation


A Key Paper• Stuart Beal (1941-2006), a leading

figure in the development of NONMEM®, considered the problem in 2001

• He identified several possible approaches

– M1: discard values below limit and estimate as if nothing had happened

– M2: treat the remaining observations as coming from a truncated distribution and use maximum likelihood

– M3: include the information from the missing observations by treating them as censored and use maximum likelihood

– M4: as for M3 but add condition that values must be greater than zero

– M5 & M7 replace censored values by LLOQ/2 or 0 respectively

– M6 as M5 but when measurements are taken over time discard any after the first below the limit

Citations to Beal, 2001, by year according to Web of Knowledge

232 citations by 6 December 2015

Probability v Likelihood

Probability• Parameters are assumed

fixed• Outcomes are variable• The ‘space’ is the space

of possible outcomes

• Probability over all possible outcomes will sum (integrate) to 1

Likelihood• Parameters are allowed

to vary• Outcomes are fixed• The ‘space’ is the space

of possible parameter values

• Likelihood over all possible parameter values will not sum (integrate) to 1


Probability and Likelihood


Method of Maximum Likelihood

• Due to RA Fisher (1890-1962)• Express the probability of the observed

result as a function of the possible parameter values

• Call this likelihood• Choose that value of the parameter that

maximises the likelihood



Purpose• A theoretical investigation of the performance of

M2 and M3– Various authors (including Beal) have made investigations based

on simulation

• Draw some connections– Earlier work on truncated and censored sample– More recent work on missing data

• Make some (tentative) recommendations• Warning: Normality is assumed and (as the

example shows) this can be inappropriate– This will be picked up at the end


NB• I consider the simplified problem of estimating a

mean at a given time point– In reality pharmacokinetic models fit concentration

over time• As a further simplification I assume no hierarchy

– Either several individuals measured once– Or one individual measured several times

• Justification– This concentrates on the pure aspect of information

loss through censoring or truncation• May be relevant for more realistic cases

– It is a much easier problem!


It’s an old Problem• Karl Pearson (1901)

– Tetrachoric correlation

• K Pearson and Alice Lee (1908)

– Method of moments

• R A Fisher (1931)– Maximum likelihood

• A Clifford Cohen– Various papers in the 1950s– Monograph ‘Truncated and

Censored Samples’, 1991

A Clifford Cohen, JnrBorn Stone County, Mississippi 1911Died Athens, Georgia, 2000Auburn University, Michigan University then appointed to maths department Georgia 1947.

Helped establish Institute of Statistics 1958 and the Department 1964.Published 74 papers including three books.Much of his work on censored and truncated samples


An Example

10 0|0122456789 10 1|2455567899 9 2|334466778 5 3|14466 2 4|28 6 5|334679 2 6|24 3 7|128 3 8|045

Phase 3 empirical therapy antifungal study where blood plasma concentrations were assayed using HPLC refined for this compound. Values in ng/ml. Here we have set LLOQ to 1000ng/ml.


Connection to Pattern Mixture Models

212

11 1

12

1 original pdf2

probability 'censored'

truncated distribution1

; , , 1 1 marginal likelihood

; , ,1

x

Q

m n m n m

i i m

m

ii

i

f x e

F Q f x dx

f xg x

F Q

L x Q F Q F Q F Q F Q mixing

f xL x Q g x

F Q

1

3 1 21

3

conditional likelihood

; , , ; , , joint likelihood

is thus the classic full likelihood usually written directly but here as pattern mixture

m

mi

mn m

ii

truncated

L L x Q L x Q F Q f x censored

L

Because we know precisely why the data are missing, they are ‘missing at random’ in Rubin’s terminology. Assume without loss of generality that the first m observations are above the limit and the next n-m are below. Here Q is short for the lower limit of quantitation. (We have used LLOQ elsewhere.)

Beal’s M2

Beal’s M3

Notes– Cohen refers to the full information case where number of missing

values is not known as truncated.• The (log) likelihood is our L2

• If the number of missing cases is known but we ignore this information we have Beal’s M2

• As Fisher showed, method of moments and maximum likelihood are identical for this problem

– Cohen refers to the full information case where the number of values is known as censored

• The (log) likelihood is our L3

• The corresponding estimation procedure is Beal’s M3• This is also sometimes known as a Tobit model

– L1, which is the likelihood for the number of ‘missing’ values alone cannot yield unique estimates for and

• This is a well known feature of pattern mixture models• Given LLOQ then a solution is available for in terms of and vice versa


(c) Senn 2015 16

Theoretical Investigation• Without loss of generality we can consider the

case where =0, =1• Then we can consider the loss that arises as the

standardised LLOQ varies from - (zero loss) to (100% loss)

• Two losses are of interest– M3 compared to full information

• Censored compared to usual (Normal) data case– M2 compared to M3

• Truncated compared to censoredTake it to the limit 16

(c) Senn 2015 17

Implementation• Calculated expected values for second derivatives

for log-likelihood for standardised Normal with standardised limit Z=(LLOQ-)/– Fisher information for L1(mixing) and L2(truncated)

• Then we can calculate Fisher information for L3 (censored) as sum of L1 and L2

• Obtain variance of estimate of for L2 and L3 cases (Beal’s M2 and M3)

• All this with much tedious but elementary algebra and calculus with the help of Mathcad®

17Take it to the limit


NB Standardised limit of quantitation is the LLOQ expressed in terms of standard deviations from the mean.

Full = no ‘missing’ values

Truncated is Beal’s M2

Censored is Beal’s M3

Variances of mu for various types of sample4.0

-3.0 -2.0 -1.0

1.0

2.0

3.0

2.5

-1.5-2.5

3.5

1.5

Standardised limit of quantitation

Variance

FullCensoredTruncated

Type of sample

Truncated, Censored and Full Information


Note the change in vertical scale. Removal of the truncated case (Beal’s M2), which has a much higher variance permits one to magnify the vertical scale and compare censored (Beal’s M3) and full cases better.

The graph now plots the censored case compared to the case with no censoring

Variances of mu for full and censored cases

1.03

1.04

1.00

-2.5

1.01

-1.5

1.02

-3.0 -1.0-2.0

Standardised limit of quantitation

Variance

CensoredFull

Type of sample

Censored and Full Information

20

NONMEM®

$PROB Beal Method 2$INPUT ID DV MDV=CENSOR$PREDLLOQ=1000YLO=LLOQVALUE=MEAN+PPVY=VALUE + SD*EPS1

$PROB Beal Method 1$INPUT ID DV MDV=CENSOR$DATA PK50.csv$EST METHOD=COND INTER LAPLACIAN MAX=9990 NSIG=3 SIGL=9$THETA3000. ; MEAN(0,2000.,) ; SD$OMEGA 0 FIX ; PPV$SIGMA 1. FIX ; EPS1$PREDVALUE=MEAN+PPVY=VALUE + SD*EPS1

$PROB Beal Method 3 $INPUT ID DV CENSOR=DROP$PREDLLOQ=1000VALUE=MEAN+PPVIF (DV.GE.LLOQ) THEN F_FLAG=0 Y=VALUE + SD*EPS1ELSE F_FLAG=1 Y=PHI((LLOQ - MEAN)/SD)ENDIF

$PROB Beal Method 4 $INPUT ID DV CENSOR=DROP$PREDLLOQ=1000VALUE=MEAN+PPVIF (DV.GE.LLOQ) THEN F_FLAG=0 Y=VALUE + SD*EPS1ELSE F_FLAG=1 CUMD=PHI((LLOQ-MEAN)/SD) CUMDZ=PHI(-MEAN/SD) Y=(CUMD-CUMDZ)/(1-CUMDZ)ENDIF

Take it to the limit

NONMEM EstimatesOriginal Data Set


Run MEAN SDFull 3289.4 2428.6Method 1 3991.4 2211.7Method 2 (L2) 1465.5 3527.8Method 3 (L3) 3072.7 2763.8Method 4 3204.5 2480.6

Comparing Various Packages


MethodM2 Truncated M3 Censored

Package

Mathcad® 1465.5 3527.8 3072.7 2763.8

NONMEM® 1465.5 3527.8 3072.7 2763.8

SAS® 1465.5 3527.8 3072.7 2763.8

GenStat® 1465.5 3527.8 3072.7 2763.8

NB Mathcad® requires programming of likelihood functions directlyIn SAS® one can use PROC NLMIXEDIn GenStat® the FITNONLINEAR procedure can be used

(c) Senn 2015 27

Multiple Imputation• Missing value problems are often handled by

multiple imputation• This immediately raises the issue as to whether

values should be discarded anyway• Surely the original values, however poorly

measured, are superior to imputed ones• Perhaps some form of weighting according to

reliability is really needed• This is closely related to the issue of Normality

Take it to the limit 27

(c) Senn 2015 28

Normality• In our example the data are clearly not Normally

distributed– True values must be positive (Beal method 4 accounts for this)

• Temptation is to log-transform• However this assumes variability is proportional to mean

– Clearly inappropriate where there are technical measurement errors

• A more complex model may be required• NB The closely related problem of measurement of serial

dilutions is another old problem– Student studied it in 1907!

Take it to the limit 28


They are neither finite Quantities nor Quantities infinitely small, nor yet nothing. May we not call them the ghosts of departed quantities? George Berkeley, 1734.

Bishop George Berkeley1685-1753

(c) Senn 2015 30

Conclusions• When given a choice M3 is clearly superior to M2• We have not investigated other approaches but

they seem inferior– With possible exception of M4

• Complicated and extensive simulation are not needed to compare these approaches

• Using a limit of quantitation is probably a bad idea in the first place

Ghosts of Departed Quantities

30Take it to the limit

take it to the limit: quantitation, likelihood, modelling and other matters

Data & Analytics